geonode.harvesting.harvesters.arcgis ==================================== .. py:module:: geonode.harvesting.harvesters.arcgis .. autoapi-nested-parse:: Harvesters for ArcGIS based remote servers. Attributes ---------- .. autoapisummary:: geonode.harvesting.harvesters.arcgis.logger Classes ------- .. autoapisummary:: geonode.harvesting.harvesters.arcgis.ArcgisRestApiLayerType geonode.harvesting.harvesters.arcgis.ArcgisServiceType geonode.harvesting.harvesters.arcgis.ArcgisServiceResourceExtractor geonode.harvesting.harvesters.arcgis.ArcgisMapServiceResourceExtractor geonode.harvesting.harvesters.arcgis.ArcgisImageServiceResourceExtractor geonode.harvesting.harvesters.arcgis.ArcgisHarvesterWorker Functions --------- .. autoapisummary:: geonode.harvesting.harvesters.arcgis.parse_remote_url geonode.harvesting.harvesters.arcgis.get_resource_extractor geonode.harvesting.harvesters.arcgis._parse_spatial_extent Module Contents --------------- .. py:data:: logger .. py:class:: ArcgisRestApiLayerType Bases: :py:obj:`enum.Enum` Generic enumeration. Derive from this class to define new enumerations. .. py:attribute:: GROUP_LAYER :value: 'Group Layer' .. py:attribute:: FEATURE_LAYER :value: 'Feature Layer' .. py:class:: ArcgisServiceType Bases: :py:obj:`enum.Enum` Generic enumeration. Derive from this class to define new enumerations. .. py:attribute:: MAP_SERVICE :value: 'MapServer' .. py:attribute:: FEATURE_SERVICE :value: 'FeatureServer' .. py:attribute:: GEOCODE_SERVICE :value: 'GeocodeServer' .. py:attribute:: GEOPROCESSING_SERVICE :value: 'GPServer' .. py:attribute:: GEOMETRY_SERVICE :value: 'GeometryServer' .. py:attribute:: IMAGE_SERVICE :value: 'ImageServer' .. py:attribute:: NETWORK_SERVICE :value: 'NAServer' .. py:attribute:: GEODATA_SERVICE :value: 'GeoDataServer' .. py:attribute:: GLOBE_SERVICE :value: 'GlobeServer' .. py:attribute:: MOBILE_SERVICE :value: 'MobileServer' .. py:method:: has_value(value) :classmethod: .. py:function:: parse_remote_url(url: str) -> Tuple[str, Optional[str], Optional[str]] Parse the input url into the ArcGIS REST catalog URL and any service name. .. py:class:: ArcgisServiceResourceExtractor(service, resource_name_filter: Optional[str] = None) Bases: :py:obj:`abc.ABC` Abstract base class with the methods that must be reimplemented in order to add support for additional ArcGIS REST services .. py:attribute:: resource_name_filter :type: Optional[str] .. py:attribute:: service :type: Type .. py:method:: get_num_resources() -> int :abstractmethod: Return the number of resources that can be extracted from the service. .. py:method:: list_resources() -> List[geonode.harvesting.harvesters.base.BriefRemoteResource] :abstractmethod: Return a list of BriefRemoteResource with the resources exposed by the service .. py:method:: get_resource(harvestable_resource: geonode.harvesting.models.HarvestableResource) -> geonode.harvesting.harvesters.base.HarvestedResourceInfo :abstractmethod: Parse the remote resource into a HarvestedResourceInfo .. py:method:: _is_relevant_layer(layer_name: str) -> bool .. py:class:: ArcgisMapServiceResourceExtractor(service: arcrest.MapService) Bases: :py:obj:`ArcgisServiceResourceExtractor` Abstract base class with the methods that must be reimplemented in order to add support for additional ArcGIS REST services .. py:attribute:: service :type: arcrest.MapService .. py:attribute:: http_session :type: requests.Session .. py:attribute:: _cached_resources :type: Optional[List[geonode.harvesting.harvesters.base.BriefRemoteResource]] .. py:method:: get_num_resources() -> int Return the number of resources that can be extracted from the service. .. py:method:: list_resources() -> List[geonode.harvesting.harvesters.base.BriefRemoteResource] Return a list of BriefRemoteResource with the resources exposed by the service .. py:method:: get_resource(harvestable_resource: geonode.harvesting.models.HarvestableResource) Parse the remote resource into a HarvestedResourceInfo .. py:method:: _extract_resources() -> List[geonode.harvesting.harvesters.base.BriefRemoteResource] .. py:method:: _list_sub_layers(arc_layer: arcrest.MapLayer) -> List[geonode.harvesting.harvesters.base.BriefRemoteResource] .. py:method:: _get_resource_descriptor(layer_representation: Dict, harvestable_resource: geonode.harvesting.models.HarvestableResource) -> geonode.harvesting.resourcedescriptor.RecordDescription .. py:method:: _parse_brief_layer(arc_layer: arcrest.MapLayer) -> geonode.harvesting.harvesters.base.BriefRemoteResource .. py:class:: ArcgisImageServiceResourceExtractor(service: arcrest.ImageService) Bases: :py:obj:`ArcgisServiceResourceExtractor` Abstract base class with the methods that must be reimplemented in order to add support for additional ArcGIS REST services .. py:attribute:: service :type: arcrest.ImageService .. py:attribute:: http_session :type: requests.Session .. py:method:: get_num_resources() -> int Return the number of resources that can be extracted from the service. .. py:method:: list_resources() -> List[geonode.harvesting.harvesters.base.BriefRemoteResource] Return a list of BriefRemoteResource with the resources exposed by the service .. py:method:: get_resource(harvestable_resource: geonode.harvesting.models.HarvestableResource) -> geonode.harvesting.harvesters.base.HarvestedResourceInfo Parse the remote resource into a HarvestedResourceInfo .. py:method:: _get_resource_name() .. py:method:: _get_resource_descriptor(layer_representation: Dict, harvestable_resource: geonode.harvesting.models.HarvestableResource) -> geonode.harvesting.resourcedescriptor.RecordDescription .. py:function:: get_resource_extractor(resource_unique_identifier: str) -> Optional[ArcgisServiceResourceExtractor] A factory for instantiating the correct extractor for the resource .. py:class:: ArcgisHarvesterWorker(remote_url: str, harvester_id: int, harvest_map_services: bool = True, harvest_image_services: bool = True, resource_name_filter: Optional[str] = True, service_names_filter: Optional[List[str]] = None) Bases: :py:obj:`geonode.harvesting.harvesters.base.BaseHarvesterWorker` Base class for harvesters. This provides two relevant things: - An interface that all custom GeoNode harvesting classes must implement; - Default implementation for common functionality. .. py:attribute:: harvest_map_services :type: bool .. py:attribute:: harvest_image_services :type: bool .. py:attribute:: resource_name_filter :type: Optional[str] .. py:attribute:: service_names_filter :type: Optional[List[str]] .. py:attribute:: http_session :type: requests.Session .. py:attribute:: _arc_catalog :type: Optional[arcrest.Catalog] .. py:attribute:: _relevant_service_extractors :type: Optional[List[Union[ArcgisMapServiceResourceExtractor, ArcgisImageServiceResourceExtractor]]] .. py:attribute:: _supported_service_types .. py:property:: allows_copying_resources :type: bool Whether copying remote resources is implemented by this worker .. py:property:: arc_catalog .. py:method:: from_django_record(harvester: Harvester) :classmethod: Return a new instance of the worker from the django harvester .. py:method:: get_extra_config_schema() -> Optional[Dict] :classmethod: Return a jsonschema schema to be used to validate models.Harvester objects .. py:method:: get_num_available_resources() -> int Return the number of available resources on the remote service. If there is a problem retrieving the number of available resource, this method shall raise `HarvestingException`. .. py:method:: list_resources(offset: Optional[int] = 0) -> List[geonode.harvesting.harvesters.base.BriefRemoteResource] Return a list of resources from the remote service. If there is a problem listing resource, this method shall raise `HarvestingException`. .. py:method:: check_availability(timeout_seconds: Optional[int] = 5) -> bool Check whether the remote service is online .. py:method:: get_geonode_resource_type(remote_resource_type: str) -> Type Return the GeoNode type that should be created from the remote resource type .. py:method:: get_geonode_resource_defaults(harvested_info: geonode.harvesting.harvesters.base.HarvestedResourceInfo, harvestable_resource: geonode.harvesting.models.HarvestableResource) -> Dict Extract default values to be used by resource manager when updating a resource .. py:method:: get_resource(harvestable_resource: geonode.harvesting.models.HarvestableResource) -> Optional[geonode.harvesting.harvesters.base.HarvestedResourceInfo] Harvest a single resource from the remote service. The return value is an instance of `HarvestedResourceInfo`. It stores an instance of `RecordDescription` and additionally whatever type is required by child classes to be able to customize resource creation/update on the local GeoNode. Note that the default implementation of `update_geonode_resource()` only needs the `RecordDescription`. The possibility to return additional information exists solely for extensibility purposes and can be left as None in the simple cases. .. py:method:: _get_extractor_class(service_type: ArcgisServiceType) -> Optional[Type] .. py:method:: _get_service_extractors(service) -> List .. py:method:: _get_relevant_services() -> List[Union[ArcgisMapServiceResourceExtractor, ArcgisImageServiceResourceExtractor]] .. py:function:: _parse_spatial_extent(raw_extent: Dict) -> Tuple[str, django.contrib.gis.geos.Polygon]