Docs menu

Data & IoT Fabric

Goal: Give your workspace its own place for device data: where it arrives, where it is stored, and where it is queried.

"One place your data is stored and queried, moved in securely and at high scale from devices, hardware or software, for IoT and edge workflows." The fabric is an Azure IoT Hub your devices connect to, and an Azure Data Explorer cluster and database that store the telemetry and answer queries, all in your own Azure subscription. Warm queries read device telemetry from this database, so a workspace needs its fabric before its device data can be queried.

Before you start

  • A connected cloud and a provisioned private landing zone. Until both exist the menu has no Data & IoT Fabric. The fabric takes its region from the landing zone.

What the window shows

There are no tabs; the window is one column, top to bottom:

PartWhat it is
Latest deploymentThe newest save from this window (yours or a teammate's), followed until it ends
What is running nowWith nothing running, the health of what this window built
Proposal rowWhen an agent has proposed fabric settings: what it proposes, with Review and Reject Proposal
Finish setup boxWhen a setup piece was left unfinished: one sentence each, and Finish setup
Data Explorer"Stores your data and answers queries on it." The cluster's state, and Start Cluster or Stop Cluster
Data pathsHot path, Warm path and Cold path
IoT Hub"Where your devices connect and send telemetry." The hub, and a warning near its daily quota
ManageEvery setting: what it is now, and what changing it does
HistoryThis window's past saves

Cards marked Coming - Stream Analytics and Functions transformations, device provisioning (DPS), Edge modules, enrollment groups and tunnels - are not built.

Set it up the first time

  1. Open Environment -> Data & IoT Fabric. Before the first save, the Manage section is the setup form, marked First save.
  2. Check IoT resource group. It starts as your landing zone's resource group name with -iot added; change it now if you want another name, because it is locked after the fabric is created.
  3. Leave the other settings or change them. The form starts on Live data arrival, 90 days of telemetry, S1 with one hub unit, the first Dev cluster tier, and the cold path and Cluster auto-stop off.
  4. Read This save will, then click Save & Provision.

The settings

"Nothing changes in Azure until you Save & Provision." Before the first save the Manage section is the setup form; after it, settings are read-only until you click Edit settings.

SettingWhat it does
IoT resource groupThe Azure resource group that holds the whole fabric, apart from the landing zone. Set before the first save; then locked
RegionTaken from your landing zone. Fixed
Hub tierS1 (400,000 messages a day per unit), S2 (6 million) or S3 (300 million). Changed in place; device traffic keeps flowing
Hub unitsA whole number: 1 to 200 on S1 and S2, 1 to 10 on S3. Billing follows tier times units
Cluster tierSix tiers: two Dev tiers with no SLA, and four Production tiers of two nodes. The list is narrowed to what Azure offers this cluster
Data arrivalLive, queryable in seconds; or Bulk, queryable in minutes, for much higher sustained volume
Keep telemetry for30 days, 90 days, 1 year or 7 years. Shortening asks first, and older telemetry is not removed straight away
Cluster auto-stopOff by default. On: Azure stops the cluster after 5 days with no data ingestion and no queries. It does not restart on its own
Hot pathAlways on. The live event stream from your hub
Warm pathAlways on. Telemetry lands in your Data Explorer database, where every query reads it
Cold pathOff until you turn it on. A private storage account and capture container for a copy of your telemetry. Once the storage account exists, the cold path stays on
Notes for later wiringYour reminders. A save stores them; nothing acts on them

The cold path's storage bills for what it holds. Capturing telemetry into it is switched on separately, and is off by default.

Before you save, This save will lists every create, change and permanent change, and the button reads Save & Provision (N changes). With nothing changed, it says "Nothing has changed yet." Some changes ask you to confirm first, in this order: shortening retention, changing the cluster tier, changing the hub's scale.

Save & Provision

Save & Provision saves the settings and builds the fabric in your Azure subscription: "the IoT resource group, the hub and its consumer groups, the Data Explorer cluster and database, and the cold storage when the cold path is on." It does not wait for Deploy.

Saved. Provisioning has been queued — the status above shows its progress and result.

The window follows the save in Latest deployment and you can close it; reopening it picks the save up again. A save that fails is rolled back: the settings are not stored, and anything already created in Azure is left in place. A save made while the cluster is stopped starts it, and it bills until stopped. A tier, auto-stop, arrival or retention change made while the cluster is not running is stored, and the next save made while it runs applies it.

Finish setup is the same save, offered when a setup piece was left unfinished. Each unfinished piece also adds one to the menu badge.

NOTE

What the fabric creates bills to your Azure subscription: the hub by its tier and units, the Data Explorer cluster while it runs, and the cold storage account for what it holds.

Start and Stop the cluster

Start Cluster and Stop Cluster act at once and are not part of a save. Start does not ask and names its charge. Stop asks first:

Stop the analytics cluster? Queries and dashboards will stop working, and compute charges stop with them. Your hub keeps collecting telemetry and holds it for 7 days, so starting again within a week loses nothing.

A stopped cluster puts a badge on the menu item, because no query works while it is down. If the cluster has to be rebuilt, Save & Provision offers the rebuild as a confirm; a rebuild deletes the cluster and its database and everything stored there, and needs the Workspace.Deploy right.

The hub's daily quota

At 80% or more of today's message limit, the IoT Hub card warns, the menu item shows a badge, and the window offers a suggested scale that opens Edit settings with it filled in. At the limit, Azure stops accepting device messages until the daily count resets.

Save and Deploy

ActionWhen it takes effect
Save & Provision, Finish setupAt once, in Azure
Start Cluster, Stop ClusterAt once, and not part of a save
Removing the fabricNot from this window

Rights

Save & Provision checks no access right beyond being in the workspace. A rebuild needs Workspace.Deploy.

Azi and agents

Azi and agents can propose the fabric's settings: name, description, data arrival, retention, the paths, cluster tier, auto-stop, hub tier and units, and notes. The resource group and region only before the fabric is provisioned. A proposal may lengthen retention, never shorten it. A workspace has one pending fabric proposal at a time.

A pending proposal shows in the window as "An agent has proposed data fabric settings for this workspace":

  • Review opens the proposed values in Manage, and the button becomes Accept Proposal and Save: it accepts the proposal first, then provisions it. "A pending proposal is not included in this Save until you press Review."
  • Reject Proposal asks "Reject this proposal? Nothing is created in Azure, and every change in its bundle is rejected."

"Accepting takes the whole proposal bundle it came in. Changes of other kinds in that bundle are accepted too and are not listed here." When the window cannot act on a proposal - it names no fabric, or it removes the fabric - it offers no Review and says instead: "Review and accept it in the Proposals panel. Nothing below changes until you do."

Messages you may see

CodeMessage or meaning
PRIVATE_LANDING_ZONE_REQUIRED"This needs a private landing zone, which this workspace does not have yet."
INVALID_RESOURCE_GROUP_NAMEThe resource group must be 1 to 90 letters, numbers, underscores, parentheses, hyphens or periods, and may not end in a period
CREATE_TIME_ONLY_FIELDThe resource group and region cannot change once the fabric is provisioned
COLD_ONE_WAY"Cold storage is provisioned for this workspace, and it cannot be turned off once it is."
RETENTION_SHORTENEDA proposal tried to shorten retention. A person shortens it, after a confirm
TELEMETRY_REQUIRES_PRIVATE_IOTA query for telemetry before this fabric exists: "Set up IoT for this workspace first."
Rebuild without the right"Rebuilding the analytics cluster requires the 'Workspace.Deploy' access right, which this identity does not hold. Grant it in Access Management."
A failed saveThe save did not complete and the platform rolled the record back; the reason follows
Watching stoppedThe window stopped watching after about an hour; provisioning may still be running - reopen it to check

Next steps

On this page