Reference

container = AzContainer("containername"; storageaccount="myacccount", kwargs...)

container is a handle to a new or existing Azure container in the myaccount sorage account. The storage account must already exist.

Additional keyword arguments

• session=AzSession(;lazy=false,scope=offline_access+openid+https://storage.azure.com/user_impersonation) user credentials (see AzSessions.jl package).
• nthreads=Sys.CPU_THREADS number of system threads that OpenMP will use to thread I/O.
• connect_timeout=30 client-side timeout for connecting to the server.
• read_timeout=10 client-side timeout for receiving the first byte from the server.
• nretry=10 number of retries to the Azure service (when Azure throws a retryable error) before throwing an error.
• verbose=0 verbosity flag passed to libcurl.

Notes

The container name can contain "/"'s. If this is the case, then the string preceding the first "/" will be the container name, and the string that remains will be pre-pended to the blob names. This allows Azure to present blobs in a pseudo-directory structure. Note that trailing and leading /'s are ignored.

containers(;storageaccount="mystorageaccount", session=AzSession(;lazy=false, scope=__OAUTH_SCOPE), nretry=5, verbose=0, connect_timeout=30, read_timeout=10)

list all containers in a given storage account.

cp(from..., to...)

copy a blob to a local file, a local file to a blob, or a blob to a blob.

Examples

local file to blob

cp("localfile.txt", AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob.txt"; buffersize=2_000_000_000, show_progress=false)

blob to local file

cp(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob.txt", "localfile.txt", buffersize=2_000_000_000, show_progress=false)

buffersize is the memory buffer size (in bytes) used in the copy algorithm, and defaults to 2_000_000_000 bytes (2GB). Note that half of this memory is used to buffer reads, and the other half is used to buffer writes. Set show_progress=true to display a progress bar for the copy operation.

blob to blob

cp(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob_in.txt", AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob_out.txt")

cp(from, to)

copy a blob to a local file, a local file to a blob, or a blob to a blob.

Examples

local file to blob

cp("localfile.txt", open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob.txt"); buffersize=2_000_000_000, show_progress=false)

blob to local file

cp(open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob.txt"), "localfile.txt"; buffersize=2_000_000_000, show_progress=false)

buffersize is the memory buffer size (in bytes) used in the copy algorithm, and defaults to 2_000_000_000 bytes (2GB). Note that half of this memory is used to buffer reads, and the other half is used to buffer writes. Set show_progress=true to display a progress bar for the copy operation.

blob to blob

cp(open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob_in.txt"), open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob_out.txt"))

cp(container_src, container_dst)

copy container_src::AzContainer and its blobs to container_dst::AzContainer.

dirname(container)

Returns the name of the Azure container that container::AzContainer is a handler to.

isdir(container)

Returns true if container::AzContainer exists.

mkpath(container)

create an Azure container from the handle container::AzContainer. If the container already exists, then this is a no-op.

readdir(container)

list of objects in a container.

rm(container)

remove container::AzContainer and all of its blobs.

cp(from..., to...)

copy a blob to a local file, a local file to a blob, or a blob to a blob.

Examples

local file to blob

cp("localfile.txt", AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob.txt"; buffersize=2_000_000_000, show_progress=false)

blob to local file

cp(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob.txt", "localfile.txt", buffersize=2_000_000_000, show_progress=false)

buffersize is the memory buffer size (in bytes) used in the copy algorithm, and defaults to 2_000_000_000 bytes (2GB). Note that half of this memory is used to buffer reads, and the other half is used to buffer writes. Set show_progress=true to display a progress bar for the copy operation.

blob to blob

cp(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob_in.txt", AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob_out.txt")

cp(from, to)

copy a blob to a local file, a local file to a blob, or a blob to a blob.

Examples

local file to blob

cp("localfile.txt", open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob.txt"); buffersize=2_000_000_000, show_progress=false)

blob to local file

cp(open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob.txt"), "localfile.txt"; buffersize=2_000_000_000, show_progress=false)

buffersize is the memory buffer size (in bytes) used in the copy algorithm, and defaults to 2_000_000_000 bytes (2GB). Note that half of this memory is used to buffer reads, and the other half is used to buffer writes. Set show_progress=true to display a progress bar for the copy operation.

blob to blob

cp(open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob_in.txt"), open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "remoteblob_out.txt"))

deserialize(container, "blobname")

read and deserialize from a blob "blobname" in container::AzContainer.

Example

io = open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "foo.bin")
serialize(io, (rand(10),rand(20)))
a,b = deserialize(io)

deserialize(object)

read and deserialize a blob object::AzObject. See deserialize(container, "blobname").

filesize(container, "blobname")

Returns the size of the blob "blobname" that is in container::AzContainer

filesize(object::AzObject)

Returns the size of the blob corresponding to object::AzObject

isfile(container, "blobname")

Returns true if the blob "object" exists in container::AzContainer.

isfile(object::AzObject)

Returns true if the blob corresponding to object exists.

joinpath(container, blobname) -> AzObject

Create a handle to an Azure blob with the name blobname::String in the Azure storage container: container::AzContainer.

Example:

io = joinpath(AzContainer("mycontainer"; storageaccount="myaccount"), "foo.bin")
write(io, rand(10))

open(container, blobname) -> AzObject

Create a handle to an Azure blob with the name blobname::String in the Azure storage container: container::AzContainer.

Example:

io = open(AzContainer("mycontainer"; storageaccount="myaccount"), "foo.bin")
write(io, rand(10))

open(object::AzObject[, mode="w+"]) -> object

This is an identity operation to support compatability with POSIX I/O. It allows for the following equivalence which can be useful in building methods that are agnostic to storage systems:

io = open(joinpath(AzContainer("foo";storageaccount="bar"), "bar")) # Azure blob sorage
io = open(joinpath("foo", "bar")) # POSIX
write(io, "hello")
close(io)

Please note that the 'mode' is for compatability with Base.open and does not have any effect due to the how Azure blob storage works.

read(container, "blobname", String)

returns the contents of the blob "blobname" in container::AzContainer as a string.

read(object, String)

read a string from object::AzObject.

Example

io = open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "foo.txt")
read(io, String)

read!(container, "blobname", data; offset=0)

read from the blob "blobname" in container::AzContainer into data::DenseArray, and where offset specifies a number of bytes in the blob to skip before reading. This method returns data. For example,

data = read!(AzContainer("foo";storageaccount="bar"), "baz.bin", Vector{Float32}(undef,10))

read!(object, x; offset=0) -> x

read data from object::AzObject into x::DenseArray, and return x. offset is an integer that can be used to specify the first byte in the object to read.

Example

io = open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "foo.txt")
read!(io, Vector{Float64}(undef, 10))

readdlm(container, "blobname", args...; options...)

Read the data in a delimited blob with the name blobname in container container::AzContainer

readdlm(io:AzObject, args...; options...)

return the parsed delimited blob from the io object io::AzObject

Example

io = open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "foo.txt")
data = readdlm(io)

rm(container, "blobname")

remove the blob "blobname" from container::AzContainer.

rm(object::AzObject; force=false)

remove the blob corresponding to object::AzObject. Note that the force keyword argument does not change the behavior of this method. It is included to match Julia's Base.rm method, allowing the calling code to work on both POSIX and Azure storage.

serialize(container, "blobname", data)

Serialize and write data to a blob with the name blobname in container::AzContainer.

Example

container = AzContainer("mycontainer";storageaccount="mystorageaccount")
serialize(container, "foo.bin", (rand(10),rand(20)))
a,b = deserialize(io)

serialize(io::AzObject, data)

Serialize and write data to io::AzObject. See serialize(conainer, blobname, data).

write(container, "blobname", data::AbstractString; contenttype="text/plain")

Write the string data to a blob with name blobname in container::AzContainer. Optionally, one can specify the content-type of the blob using the contenttype keyword argument. For example: content-type="text/plain",content-type="applicaton/json", etc..

write(container, "blobname", data::StridedArray)

Write the array data to a blob with the name blobname in container::AzContainer.

write(io::AzObject, data)

write data to io::AzObject.

Example

io = open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "foo.bin")
write(io, rand(10))
x = read!(io, zeros(10))

writedlm(container, "blobname", data, args...; options...)

Write the array data to a delimited blob with the name blobname in container container::AzContainer

writedlm(io:AzObject, data, args...; options...)

write the array data to io::AzObject

Example

io = open(AzContainer("mycontainer";storageaccount="mystorageaccount"), "foo.txt")
writedlm(io, rand(10,10))
x = readdlm(io)