buffer

The buffer command is used to create and/or populate one of these named buffers with data.


Syntax

buffer_name=protocol protocol_parameter(s)

Details

The first argument to the buffer statement is the name of the buffer to create. If a buffer with this name already exists then any data it contains will be overwritten.

There must be whitespace on both sides of the ‘equals’ symbol following the buffer name.

The following protocols are supported:

file

buffer_buffername= filefilename

The file protocol imports a file directly into a buffer. This can be very useful when developing USE scripts, as the USE script for processing for a JSON file (for example) can be implemented without requiring access to a server.

If the specified buffer name already exists, then a warning will be logged and any data in it will be cleared before importing the file.

data

buffer_buffername= datastring

The data protocol populates the buffer with the literal text specified in string. This is useful when extracting embedded JSON. For example the JSON snippet below contains embedded JSON in the instanceData field:

"properties": {
"subscriptionId":"sub1.1",
"usageStartTime": "2015-03-03T00:00:00+00:00",
"usageEndTime": "2015-03-04T00:00:00+00:00",
"instanceData":"{\"Microsoft.Resources\":{\"resourceUri\":\"resourceUri1\",\"location\":\"Alaska\",\"tags\":null,\"additionalInfo\":null}}", "
quantity"
:2.4000000000,
"
meterId"
:"meterID1"
}

In this case, the instanceData field can be extracted using a parslet, placed into a new buffer and re-parsed to extract the values within it. Assuming the snippet is in a file called my_data.json, this would be done as follows:

buffer properties = file my_data.json
          

var instanceData = $JSON{my_data}.[properties].[instanceData]

buffer embedded = data ${instanceData} print The embedded resourceUri is $JSON{embedded}.[Microsoft.Resources].[resourceUri]

http

buffer_buffername= httpmethod url

For full details on the HTTP protocol and its parameters, please refer to the http article.

Once the HTTP request has been executed, any data it returned will be contained in the named buffer, even if the data is binary in format (EX: images, audio files, or anything else non-human readable).

If the HTTP request returned no data, one of the following will apply:

  • If the buffer does not already exist then the buffer will not be created
  • If the buffer already exists then it will be deleted altogether

For details of how to access the data in a named buffer, please refer to the USE script basics article.

odbc

buffer_buffername= odbcdsn [username password] query

Username and password are optional, but neither or both must be specified.

where:

  • dsn is the ODBC Data Source Name (this should be configured at the OS level)
  • username and password are the credentials required by the DSN
  • query is an SQL query

Once the query has been executed, the resulting data is located in the named buffer. It can subsequently be saved as a CSV file to disk using:

save {buffername} as filename.csv

The resulting CSV uses a comma (,) as the separator and double quotes (") as the quoting character. Any fields in the data which contain a comma will be quoted.

odbc_direct

buffer_buffername= odbc_directquery

where query is an SQL query.

Executes SQL query against ODBC datasource that is described in the set's odbc_connect parameter.

Once the query has been executed, the resulting data is located in the named buffer. It can subsequently be saved as a CSV file to disk using:

save {buffername} as filename.csv

The resulting CSV uses a comma (,) as the separator and double quotes (") as the quoting character. Any fields in the data which contain a comma will be quoted.


Examples

The following examples retrieve data from ODBC and HTTP sources:

# Typical usage in USE script to retrieve all data from the usage table
          
          buffer odbc_csv = odbc ExivityDB admin secret "select * from usage"
          save {odbc_csv} as "odbc.csv"
          discard {odbc_csv}
          
          # Retrieve the service summary from a local CloudCruiser 4 server and place it in a buffer
          set http_username admin
          set http_password admin
          set http_authtype basic
          
          buffer services = http GET "http://localhost:8080/rest/v2/serviceCatalog/summaries"
          # The 'services' buffer now contains the HTTP response data

How did we do?