Azure Event Hubs Capture walkthrough. Event Hubs Capture is a feature of Event Hubs that enables you to automatically deliver the streaming data in your event hub to an Azure Blob storage account of your choice. This capability makes it easy to perform batch processing on real- time streaming data. This article describes how to use Event Hubs Capture with Python. For more information about Event Hubs Capture, see the overview article. This sample uses the Azure Python SDK to demonstrate the Capture feature. The sender. py program sends simulated environmental telemetry to Event Hubs in JSON format. The event hub is configured to use the Capture feature to write this data to blob storage in batches. The capturereader. What will be accomplished. Create an Azure Blob Storage account and a blob container within it, using the Azure portal. Create an Event Hub namespace, using the Azure portal. Create an event hub with the Capture feature enabled, using the Azure portal. Eine der umfangreichsten Listen mit Dateierweiterungen. Erweiterung Was; 000 (000-600) Paperport Scanned Image: 000 (000-999) ARJ Multi-volume Compressed Archive. Send data to the event hub with a Python script. Read the files from the capture and process them with another Python script. Prerequisites. Create an Azure Storage account. Log on to the Azure portal. In the left navigation pane of the portal, click New, then click Storage, and then click Storage Account. Complete the fields in the storage account blade and then click Create. After you see the Deployments Succeeded message, click the name of the new storage account and in the Essentials blade, click Blobs. When the Blob service blade opens, click + Container at the top. Name the container capture, then close the Blob service blade. Click Access keys in the left blade and copy the name of the storage account and the value of key. Save these values to Notepad or some other temporary location. Create a Python script to send events to your event hub. Open your favorite Python editor, such as Visual Studio Code. Create a script called sender. This script sends 2. They are simple environmental readings sent in JSON. Paste the following code into sender. Service. Bus. Service. Service. Bus. Service(service_namespace='INSERT YOUR NAMESPACE NAME', shared_access_key_name='Root. Manage. Shared. Access. Key', shared_access_key_value='INSERT YOUR KEY').
INSERT YOUR EVENT HUB NAME', s). Update the preceding code to use your namespace name, key value, and event hub name that you obtained when you created the Event Hubs namespace. Create a Python script to read your Capture files. Fill out the blade and click Create. Create a script called capturereader. This script reads the captured files and creates a file per device to write the data only for that device. Paste the following code into capturereader. Data. File. Reader, Data. File. Writer. from avro. Datum. Reader, Datum. Writer. from azure. Block. Blob. Service. Blob(filename). reader = Data. File. Reader(open(filename, 'rb'), Datum. Reader()). for reading in reader. Body"]). if not 'id' in parsed_json. File = open(device + '. File. write(", ". Processing(account. Name, key, container). Processor started using path: ' + os. Block. Blob. Service(account_name=account. Name, account_key=key). Downloaded a non empty blob: ' + blob. Name = string. replace(blob. Name). process. Blob(clean. Name). os. remove(clean. Name). block_blob_service. Processing('YOUR STORAGE ACCOUNT NAME', 'YOUR KEY', 'capture'). Be sure to paste the appropriate values for your storage account name and key in the call to start. Processing. Run the scripts. Open a command prompt that has Python in its path, and then run these commands to install Python prerequisite packages: pip install azure- storage. If you have an earlier version of either azure- storage or azure, you may need to use the - -upgrade option. You might also need to run the following (not necessary on most systems): pip install cryptography. Change your directory to wherever you saved sender. This command starts a new Python process to run the sender. Now wait a few minutes for the capture to run. Then type the following command into your original command window: python capturereader. This capture processor uses the local directory to download all the blobs from the storage account/container. It processes any that are not empty, and writes the results as . Next steps. You can learn more about Event Hubs by visiting the following links. All Attributes (Windows).
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
November 2017
Categories |