The Update a Feature in a Spatiotemporal Big Data Store output can be used to update feature records in a spatiotemporal big data store with attributes obtained from processed event records. This type of output only supports updating existing feature records in hosted map and feature services published to an ArcGIS Enterprise portal whose hosting server has a registered spatiotemporal big data store—a data store type available with ArcGIS Data Store.
Client web mapping applications access feature records stored in a spatiotemporal big data store through hosted map and feature services that offer data aggregation and visualization capabilities specific to this type of enterprise geodatabase.
Learn more about data aggregation capability
Event records routed to this output should have an associated GeoEvent Definition with an attribute field tagged TRACK_ID. The output uses the attribute value from this tagged field to match the attribute value taken from the event record's unique feature identifier field value to identify an existing feature record to update. If an event record does not have a field tagged TRACK_ID, or the value from the tagged field cannot be matched to an existing feature record, a new feature record is created. This output assumes a one-to-one relationship between processed event records and existing feature records. If multiple feature records exist whose attribute values in the specified unique feature identifier field match the value in a processed event record's tagged TRACK_ID field, feature records may be inconsistently or incorrectly updated.
Usage notes
Keep the following in mind when working with the Update a Feature in a Spatiotemporal Big Data Store output:
- This output supports the update of feature records in a spatiotemporal big data store enterprise geodatabase.
- The output pairs a no operation outbound adapter with the spatiotemporal big data store outbound transport. All event record formatting, caching, enterprise geodatabase connection, and feature creation operations are managed by the outbound transport.
- Client web mapping applications access feature records stored in a spatiotemporal big data store through hosted map and feature services.
- ArcGIS GeoEvent Server must have a registered server connection whose type is Enterprise, also known as a portal type connection, to obtain connection information from an Enterprise portal's registered spatiotemporal big data store. A direct connection is made from GeoEvent Server to the spatiotemporal big data store enterprise geodatabase to add or update feature records in a spatiotemporal big data store.
- The Register ArcGIS Server capability, available when configuring a new output, allows you to register a connection to a stand-alone ArcGIS Server installation, an Enterprise hosting server, or the hosting server used by your ArcGIS Online organization. Only an Enterprise type connection, however, allows the output to interface with a spatiotemporal big data store.
- The Create Data Source capability, available when configuring a new output, allows you to create a spatiotemporal big data store data source. As with publishing a new feature service, the ability to create a new data source requires an existing GeoEvent Definition that reflects the attribute fields and schema you want feature records to exhibit. Note that the GeoEvent Definition you use must honor restrictions imposed by this type of enterprise geodatabase, and the data source schema cannot be changed once it has been created. Unlike traditional map or feature services, multiple hosted map and feature services can be published that use the same spatiotemporal big data store data source. These hosted map or feature services provide unique data aggregation and visualization capabilities as well as RESTful access to the feature records held in the spatiotemporal big data store.
- Any spatiotemporal big data store data source you create using GeoEvent Manager is initially empty. Use GeoEvent Server to add or update feature records in the data source with data from real-time feeds processed through a GeoEvent Service.
- A spatiotemporal big data store data source cannot be created for an ArcGIS Online organization or an ArcGIS Server that is not an Enterprise portal's hosting server.
Learn more about working with the spatiotemporal big data store in GeoEvent Server
- There are two parameters available to suppress, or skip, feature record updates when data older than an existing feature record is received.
- The Skip write when the feature is older for a single writer parameter applies when exactly one writer is used to update feature records.
- The Skip write when the feature is older for multiple writers parameter supports scenarios where updates may be performed by multiple writers.
- When neither the Skip write when the feature is older for a single writer parameter nor the Skip write when the feature is older for multiple writers parameter is selected, GeoEvent Server defaults to updating existing feature records every time sensor data is received and processed. This can result in data from old observations received out of temporal order being used to update existing feature records.
- Choose the Skip write when the feature is older for a single writer parameter when all of the following are true:
- Exactly one output for Update a Feature in a Spatiotemporal Big Data Store is configured.
- The output’s Write Concurrency Level parameter is changed from its default value of 3 concurrent writers to 1.
- Choose the Skip write when the feature is older for multiple writers parameter when either of the following conditions are true:
- Two or more outputs for Update a Feature in a Spatiotemporal Big Data Store are configured.
- A single output is configured with the default Write Concurrency Level parameter set of 3 concurrent writers.
- When both the Skip write when the feature is older for a single writer and Skip write when the feature is older for multiple writers parameters are chosen, multiple writers scenario is assumed.
- When the Skip write when the feature is older for a single writer parameter or the Skip write when the feature is older for multiple writers parameter is selected to suppress, or skip, feature record updates, you must specify a feature record attribute to use as a comparison key using the Skip write when the feature is older based on this field parameter.
- The specified field name must exactly match an attribute field name in the spatiotemporal big data store feature record schema.
- The specified attribute field may contain a date and time, a string, or a numeric value (such as, a 32-bit integer or a 64-bit floating-point value).
- When the specified attribute field in a processed data record holds a null value, GeoEvent Server discards the processed data record to prevent the update of an existing feature record.
- When a date and time value is used as a comparison key, the following happens:
- If the processed data record’s timestamp is equal to or later than the timestamp of the existing feature record, GeoEvent Server updates the existing feature record.
- If the processed data record’s timestamp is earlier than the timestamp of an existing feature record, GeoEvent Server discards the processed data record to suppress, or skip, the feature record update, preventing data from an older observation from being used to update an existing feature record.
- When a numeric value is used as the comparison key GeoEvent Server uses a numeric comparison. When a string value is used as a comparison key, GeoEvent Server compares the value using a lexicographic (alphabetic) comparison.
- If the processed data record’s numeric value is greater than or equal to the corresponding key in the existing feature record, GeoEvent Server updates the existing feature record.
- If the processed data record’s numeric value is less than the corresponding key in the existing feature record, GeoEvent Server discards the processed data record to suppress, or skip, feature record update, preventing data from an older observation from being used to update an existing feature record.
Parameters
The following are the parameters for the Update a Feature in a Spatiotemporal Big Data Store output:
| Parameter | Description |
|---|---|
| Name | A descriptive name for the output connector used for reference in GeoEvent Manager. |
| GeoEvent Definition | The name of an existing GeoEvent Definition to use when adapting received data to create event data for processing by a GeoEvent Service. Note:The schema of the GeoEvent Definition specified must mirror the schema of the spatiotemporal big data store data source. Any fields whose name has changed (field names are case sensitive) cause the Spatiotemporal Big Data Store outbound transport to log a warning that the specified GeoEvent Definition is incompatible with the data source. When processed event records are sent to the output, an error message is logged or displayed by the output connector that one or more expected attribute fields are missing. Event records are not added or updated in the spatiotemporal big data store data source until the specified GeoEvent Definition correctly represents all expected attribute names. This includes capitalization errors such as Geometry instead of geometry. |
| Registered Server Connection | The ArcGIS Enterprise connection registered with ArcGIS GeoEvent Server as a data store. The ArcGIS Enterprise connection must have a hosting server configured with the ArcGIS Data Store enabled as a spatiotemporal big data store. Registered server connections cache information about map and feature services, their layers, and layer properties. |
| Data Source Name | The name of the data source in the spatiotemporal big data store where processed event records are saved as feature records. |
| Unique Feature Identifier Field | The attribute field in the spatiotemporal big data store's data source that contains a unique feature identifier for a sensor or trackable asset. When searching to identify a feature record to update, this output looks for a feature record whose Unique Feature Identifier Field value exactly matches the value taken from a processed event record's field bearing the TRACK_ID tag. If an event record either does not have a field tagged with the TRACK_ID tag, or the value in the tagged field cannot be found in an existing feature record's Unique Feature Identifier Field value, a new feature record is created. As future event records are processed, the record created should be updated since it now exists in the data source with a unique feature identifier. This output assumes a one-to-one relationship between processed event records and existing feature records. If multiple feature records exist whose Unique Feature Identifier Field values match the value in a processed event record's tagged TRACK_ID field, feature records may be inconsistently or incorrectly updated. Note:The parameter is required to update feature records in the spatiotemporal big data store. Ensure that the GeoEvent Definition has an attribute field tagged with the TRACK_ID tag and that attribute values of this field match the value of an existing feature record's unique feature identifier. |
| Flush Interval (millisecond) | Specifies how often (in milliseconds) feature records are updated in the designated spatiotemporal big data store data source. The default is 1000 milliseconds, which means processed event records are updated in the data source every second. |
| Maximum Features Per Transaction | The maximum number of records to include in any single feature service request to update feature records. If the outputs cache contains more than the specified maximum number of records, multiple requests are made until the outputs cache has been emptied. The default is 1000. It is recommended that you balance the size and complexity of each event record with the number of event records processed each second. When event records are particularly large, with many attribute fields or large complex geometry objects, reduce the value to avoid sending requests larger than a feature service can handle. However, depending on the speed and volume of event records being processed each second, setting a value too low may result in an overwhelming number of transaction requests. |
| Write Concurrency Level | The number of concurrent writers (or threads) that write in parallel when sending records to the spatiotemporal big data store. The spatiotemporal big data store can handle multiple threads; increased throughput can be achieved by adjusting the write concurrency level. The default is 3. The effectiveness of the Write Concurrency Level parameter depends on many factors such as the volume of data being written, the size of each event record, the speed of the network, and the compute capacity of each machine. While adjusting this parameter value can result in improved event record throughput, it should not be casually changed without considering these factors. The Flush Interval and Maximum Features Per Transaction parameter values should also be considered. |
| Skip write when the feature is older for a single writer | Suppress, or skip, feature record updates when the comparison key value in a processed data record is temporally earlier, numerically less than, or comes alphabetically before the corresponding key value in an existing feature record. Specify Yes to enable this parameter when exactly one output for Update a Feature in a Spatiotemporal Big Data Store is configured and the output’s Write Concurrency Level parameter is changed from its default value of 3 concurrent writers to 1 concurrent writer. The supported options for this parameter are Yes or No. The default is No. When both the Skip write when the feature is older for multiple writers and Skip write when the feature is older for a single writer parameters are set to No, GeoEvent Server updates feature records without comparing a specified key to determine if the processed data record is newer or older than the existing feature record. When the Skip write when the feature is older for a single writer parameter is set to Yes, GeoEvent Server compares the specified key value in each processed data record to the corresponding key value in the existing feature record before performing an update. If the processed data record is determined to be older than the existing feature record, the update is suppressed to prevent existing feature records from being overwritten with older data. |
| Skip write when the feature is older for multiple writers | Suppress, or skip, feature record updates, when the comparison key value in a processed data record is temporally earlier, numerically less than, or comes alphabetically before a corresponding key value in an existing feature record. This parameter supports scenarios in which multiple concurrent writers are used to update feature records in the spatiotemporal big data store. Specify Yes to enable this parameter when two or more outputs for Update a Feature in a Spatiotemporal Big Data Store are configured to write to the same spatiotemporal big data source or when a single output’s Write Concurrency Level parameter is set to its default of 3 concurrent writers. The supported options for this parameter are Yes or No. The default is No. When both the Skip write when the feature is older for multiple writers and Skip write when the feature is older for a single writer parameters are set to No, GeoEvent Server updates feature records without comparing a specified key to determine if the processed data record is newer or older than the existing feature record. When the Skip write when the feature is older for multiple writers parameter is set to Yes, GeoEvent Server compares the specified key value in processed data records to the corresponding key value in the existing feature record before updating the feature record, even when multiple writers write concurrently, to prevent existing feature records from being overwritten with older data. |
| Skip write when the feature is older based on this field | Specifies the attribute field name from which a comparison key value is retrieved to determine if a processed data record is newer or older than an existing feature record in the spatiotemporal big data store. The attribute field name must exactly match a field in both the processed data record schema and the feature record schema in the spatiotemporal big data store. If the specified attribute field in a processed data record contains a null value, the processed data record is discarded and not used to update an existing feature record. |