Data model
Introduction
Normally you don't need to access MongoDB directly as Orion Contex Broker uses it transparently. However, for some operations (e.g. backup, fault recovery, etc.) it is useful to know how the database is structured. This section provides that information.
In the case you need to access database directly, be very careful when manipulation it, as some actions could be irreversible (doing a backup at the beginning it's a good idea).
Orion Context Broker uses four collections in the database, described in the following subsections.
entities collection
The entities collection stores information about NGSI entities. Each document in the collection corresponds to an entity.
Fields:
- _id stores the EntityID, including the ID itself and type.
Given that we use _id for this, we ensure that EntityIDs
are unique. The JSON document for this field includes:
- id: entity NGSI ID
- type: entity NGSI type
- servicePath: related with the service path functionality.
- attrs is an keymap of the different attributes that have been
created for that entity. The key is generated with the attribute
name (changing "." for "=", as "." is not a valid character in
MongoDB document keys).
Each element in the map has the following information:
- type: the attribute type
- value: the attribute value (for those attribute that has received at least one update). Up to version 0.10.1, this value is always a string, but in 0.11.0 this value can be also a JSON object or JSON vector to represent an structured value (see section about attribute representation in Orion API specification).
- md (optional): custom metadata. This is a keymap of metadata objects. The key is generated with the metadata name (changing "." for "=", as "." is not a valid character in MongoDB document keys), e.g. a metadata with name "m.x" will use the key "m=x". The object value of each key has two fields: type and value (of the metadata).
- mdNames: an array of strings. Its elements are the names of the metadata of the attribute. Here the "." to "=" replacement is not done.
- creDate: the timestamp (as a floating point number, meaning seconds with milliseconds) corresponding to attribute creation (as a consequence of append).
- modDate: the timestamp (as a floating point number, meaning seconds with milliseconds) corresponding to last attribute update. It matches creDate if the attribute has not been modified after creation.
- attrNames: an array of strings. Its elements are the names of the attributes of the entity (without IDs). In this case, the "." to "=" replacement is not done.
- creDate: the timestamp (as a floating point number, meaning seconds with milliseconds) corresponding to entity creation date (as a consequence of append).
- modDate: the timestamp (as a floating point number, meaning seconds with milliseconds) corresponding to last entity update. Note that it is usually the same as a modDate corresponding to at least one of the attributes (not always: it will not be the same if the last update was a DELETE operation). It matches creDate if the entity has not been modified after creation.
- location (optional): geographic location of the entity, composed
of the following fields:
- attrName: the attribute name that identifies the geographic location in the attrs array
- coords: a GeoJSON representing the location of the entity. See below for more details.
- lastCorrelator: value of the root correlator in the last
update request on the entity. Used by the self-notification loop protection
logic. By root correlator we mean the value of the
Fiware-Correlator
request header in the update request without any suffix. Eg. the root correlator forFiware-Correlator: f320136c-0192-11eb-a893-000c29df7908; cbnotif=32
isf320136c-0192-11eb-a893-000c29df7908
. - expDate (optional): expiration timestamp (as a Date object) for the entity. Have a look to the transient entities functionality for more detail.
Regarding location.coords
in can use several formats:
- Representing a point:
{
"type": "Point",
"coordinates": [ -3.691944, 40.418889 ]
}
- Representing a line:
{
"type": "LineString",
"coordinates": [ [ 10, 0], [0, 10] ]
}
- Representing a polygon:
{
"type": "Polygon",
"coordinates": [ [ [ 10, 0], [0, 10], [0, 0], [10, 0] ] ]
}
- Finally,
location.coords
could hold an arbitrary JSON object, representing a location in GeoJSON format. Arbitrary GeoJSON can be used with the geo:json attribute type and it is up to the user to introduce a valid object. Note that the three above cases are actually GeoJSON representation for "fixed" cases.
Note that coordinate pairs use the longitude-latitude order, which is opposite to the order used in the Geographical Queries. This is due to the internal MongoDB geolocation implementation, (which is based in GeoJSON) uses longitude-latitude order. However, other systems closer to users (e.g. GoogleMaps) use latitude-longitude format, so we have used the latter for the Geographical Queries API.
Example document:
{
"_id":
"id": "E1",
"type": "T1",
"servicePath": "/"
},
"attrs": {
"A1": {
"type": "TA1",
"value": "282",
"creDate" : 1389376081.8471954,
"modDate" : 1389376120.2154321,
"md" : {
"customMD1": {
"type" : "string",
"value" : "AKAKA"
},
"customMD2": {
"type" : "integer",
"value" : "23232"
}
},
"mdNames": [ "customMD1", "customMD2" ]
},
"A2()ID101": {
"type": "TA2",
"value": "176",
"creDate" : 1389376244.6651231,
"modDate" : 1389376244.6651231
},
"position": {
"type": "location",
"value": "40.418889, -3.691944",
"creDate" : 1389376244.6651231,
"modDate" : 1389376244.6651231
}
},
"attrNames": [ "A1", "A2", "position" ],
"creDate": 1389376081.8471954,
"modDate": 1389376244.6651231,
"location": {
"attrName": "position",
"coords": {
"type": "Point",
"coordinates": [ -3.691944, 40.418889 ]
}
},
"lastCorrelator" : "aa01d6c6-4f7e-11e7-8059-000c29173617"
}
registrations collection
The registrations collection stores information about registrations. Each document in the collection corresponds to a registration.
Fields:
- _id is the registration ID (the value that is provided to the user to update the registration). Given that we use _id for this, we ensure that registration IDs are unique and that queries by registration IDs will be very fast (as there is an automatic default index in _id).
- format: the format to use to send forwarded requests.
For NGSIv1 format, use JSON as value for
format
. For NGSIv2, as of today, only normalized format is supported. - servicePath: related with the service path functionality.
- status (optional): either
active
(for active registrations) orinactive
(for inactive registrations). The default status (i.e. if the document omits this field) is "active". - description (optional): a free text string describing the registration. Maximum length is 1024.
- expiration: this is the timestamp (as integer number, meaning seconds) for which the registration expires.
- fwdMode: the forwarding mode supported by the provider, either:
all
,query
,update
ornone
. If omitted (Orion versions previous to 2.6.0),all
is assumed. - contextRegistration: is an array whose elements contain the
following information:
- entities: an array containing a list of
entities (mandatory). The JSON for each entity contains id,
type, isPattern and isTypePattern. Note that, due to legacy
reasons, isPattern may be
"true"
or"false"
(text) while isTypePattern may betrue
orfalse
(boolean). - attrs: an array containing a list of attributes (optional). The JSON for each attribute contains name and type.
- providingApplication: the URL of the providing application for this registration (mandatory)
- entities: an array containing a list of
entities (mandatory). The JSON for each entity contains id,
type, isPattern and isTypePattern. Note that, due to legacy
reasons, isPattern may be
Example document:
{
"_id": ObjectId("5149f60cf0075f2fabca43da"),
"format": "JSON",
"fwdMode": "all",
"expiration": 1360232760,
"contextRegistration": [
{
"entities": [
{
"id": "E1",
"type": "T1",
"isPattern": "false"
},
{
"id": "E2",
"type": "T2",
"isPattern": "false"
}
],
"attrs": [
{
"name": "A1",
"type": "TA1"
},
{
"name": "A2",
"type": "TA2"
}
],
"providingApplication": "http://foo.bar/notif"
},
"status": "active"
]
}
csubs collection
The csubs collection stores information about context subscriptions. Each document in the collection corresponds to a subscription.
Fields:
- _id is the subscription ID (the value that is provided to the user to update and cancel the subscription). Given that we use _id for this, we ensure that subscription IDs are unique and that queries by subscription IDs are very fast (as there is an automatic default index in _id).
- servicePath: related with the service
path functionality. This is the service path
associated to the query "encapsulated" by the subscription. Default
is
/#
. - expiration: this is the timestamp (as integer number, meaning seconds) on which the subscription expires. For permanent subscriptions an absurdly high value is used (see PERMANENT_SUBS_DATETIME in the source code).
- lastNotification: the time (as integer number, meaning seconds) when last notification was sent. This is updated each time a notification is sent, to avoid violating throttling.
- throttling: minimum interval between notifications. 0 or -1 means no throttling.
- reference: the URL for notifications, either HTTP or MQTT
- mqttTopic: MQTT topic (only in MQTT notifications)
- mqttQoS: MQTT QoS value (only in MQTT notifications)
- entities: an array of entities (mandatory). The JSON for each
entity contains id, type, isPattern and isTypePattern. Note that,
due to legacy reasons, isPattern may be
"true"
or"false"
(text) while isTypePattern may betrue
orfalse
(boolean). - attrs: an array of attribute names (strings) (optional).
- blacklist: a boolean field that specifies if
attrs
has to be interpreted as a whitelist (ifblacklist
is equal tofalse
or doesn't exist) or a blacklist (ifblacklist
is equal totrue
). - onlyChanged: a boolean field that specifies if only attributes that change has to be included in notifications (if onlyChanged is equal to true) or not (if onlyChanged is equal to false or doesn't exist).
- metadata: an array of metadata names (strings) (optional).
- conditions: a list of attributes that trigger notifications.
- expression: an expression used to evaluate if notifications has to be sent or not when updates come. It may be composed of the following fields: q, mq, georel, geometry and/or coords (optional)
- count: the number of notifications sent associated to the subscription.
- format: the format to use to send notification, possible values are JSON (meaning JSON notifications in NGSIv1 legacy format), normalized, keyValues, simplifiedNormalized, simplifiedKeyValues and values (the last five used in NGSIv2 format).
- status: either
active
(for active subscriptions),inactive
(for inactive subscriptions) oroneshot
(for oneshot subscriptions). Note that Orion API consider additional states (e.g.expired
) but they never hit the DB (they are managed by Orion). - statusLastChange: last time status was updated (as decimal number, meaning seconds and fractions of seconds). This is mainly to be used by the subscription cache update logic (so status in updated in DB from cache only if it is newer).
- description (optional field): a free text string describing the subscription. Maximum length is 1024.
- timeout this field configures the maximum time the subscription waits for the response for http notifications. It is a number between 0 and 1800000. If defined to 0 or omitted, the default timeout will be used.
- custom: a boolean field to specify if this subscription uses customized notifications (a functionality in the Orion API).
If this field exist and its value is "true" then customized notifications are used and the
headers
,qs
,method
andpayload
fields are taken into account. - headers: optional field to store the HTTP headers keymap for notification customization functionality.
- qs: optional field to store the query parameters keymap for notification customization functionality.
- method: optional field to store the HTTP method for notification customization functionality.
- payload: optional field to store the payload for notification customization functionality. If
its value is
null
means that no payload has to be included in the notification. If its value is""
or if the field is omitted, then the NGSIv2 normalized format is used. - json: optional field to store a JSON object or array to generated JSON-based payload for notification customization functionality in the Orion API. More detail of this functionality here
- ngsi: optional field to store a NGSI patching object for notification customization functionality
in the Orion API. More detail of this functionality here.
The value of this field is an object with a
attrs
key which value is a simplified version ofattrs
in the entities collection. - lastFailure: the time (as integer number, meaning seconds) when last notification failure occurred. Not present if the subscription has never failed.
- lastFailureReason: text describing the cause of the last failure. Not present if the subscription has never failed.
- lastSuccess: the (as integer number, meaning seconds) time when last successful notification occurred. Not present if the subscription has never provoked a successful notification.
- lastSuccessCode: HTTP code (200, 400, 404, 500, etc.) returned by receiving endpoint last time a successful notification was sent. Not present if the subscription has never provoked a successful notification.
- maxFailsLimit: An optional field used to specify the maximum limit of connection attempts, so when that number of failing notifications is reached, then the subscription passes automatically to inactive state.
- failsCounter: the number of consecutive failing notifications associated to the subscription. This is increased by one each time a notification attempt fails. It is reset to 0 if a notification attempt successes.
- altTypes: array with a list of alteration types associated to the subscription. If the field is not included, a default is assumed (check this document).
- covered: a boolean field that specifies if all
attrs
have to be included in notifications (if value is true) or only the ones existing in the triggering entity (if value is false or field is omitted). More information in covered subscription section in Orion API specification. - notifyOnMetadataChange: if
true
metadata is considered part of the value of the attribute regarding subscription triggering. Iffalse
metadata is not considered part of the value of the attribute regarding subscription triggering. Default behaviour (if omitted) is the one fortrue
.
Example document:
{
"_id" : ObjectId("5697d4d123acbf5e794ab031"),
"expiration" : NumberLong(1459864800),
"reference" : "http://localhost:1234",
"servicePath" : "/",
"entities" : [
{
"id" : ".*",
"type" : "Room",
"isPattern" : "true",
"isTypePattern": false
}
],
"attrs" : [
"humidity",
"temperature"
],
"conditions" : [ "temperature" ],
"expression" : {
"q" : "temperature>40",
"mq" : "temperature.accuracy<1",
"geometry" : "",
"coords" : "",
"georel" : ""
},
"format" : "JSON",
"status" : "active",
"statusLastChange" : 1637226173.6940024
}