Perform a NaN-aware conversion into an array. Default nan-strategy is :keep. Nan strategies can be: :keep :remove :exception

Perform a NaN-aware conversion into an array buffer. Default nan-strategy is :keep.

Options:

• :nan-strategy - Nan strategies can be: :keep :remove :exception. :remove can result in a shorter array buffer than the input data. Default is :keep.

Convert this item to a buffer implementation or throw an exception.

Efficiently convert nearly anything into a byte array.

Efficiently convert nearly anything into a char array.

Nan-aware conversion to double array. See documentation for ->array-buffer. Returns a double array. options -

• nan-strategy - :keep (default) :remove :exception

Nan-aware conversion to double array. See documentation for ->array-buffer. Returns a double array. options -

• nan-strategy - :keep (default) :remove :exception

Efficiently convert nearly anything into a int array.

Ensure this object is iterable. If item is iterable, return the item. If the item is convertible to a buffer object , convert to it and return the object. Else if the item is a scalar constant, return an infinite sequence of items.

Efficiently convert nearly anything into a long array.

If this object has a read-only or read-write conversion to a buffer object return the buffer object. Else throw an exception.

Efficiently convert nearly anything into a short array.

Copy a thing into a persistent vector.

If this item is convertible to a PrimtiveIO writing the perform the conversion and return the interface. Else throw an excepion.

Allocate an uninitialized buffer. See options for malloc.

Allocate a buffer of zeros. See options for malloc.

Return the type of a thing. Types could be: :scalar :iterable :reader :tensor (reader with more than 1 dimension)

If this item is convertible to a tech.v3.datatype.array_buffer.ArrayBuffer then convert it and return the typed buffer

If this item is or has a conversion to an implementation of the Buffer interface then return the buffer interface for this object. Else return nil.

If this item is representable as a base buffer type, either native or array, return that buffer.

If this item is convertible to a tech.v3.datatype.native_buffer.NativeBuffer return it. Return nil otherwise.

If this item is convertible to a buffer descriptor, convert it. Else return nil.

Buffer descriptors are a ND description of data. For example, a native 3x3 tensor has a buffer description like thus:

{:ptr 140330005614816
 :datatype :float64
 :endianness :little-endian
 :shape [3 3]
 :strides [24 8]}

This design allows zero-copy transfer between neanderthal, numpy, tvm, etc.

If this object has a read-only or read-write conversion to a primitive io object return the buffer object.

If this item is convertible to a PrimtiveIO writing the perform the conversion and return the interface.

Perform a checked cast of a value to specific datatype.

Clone an object. Can clone anything convertible to a reader.

Extremely fast clone assuming src is a native buffer. Exception otherwise. See options for malloc.

Coalesce data from a raw sequence of things into a contiguous buffer. Returns the buffer; uses the copy-raw->item! pathway.

Copy a sequence of blocks of countable things into a larger countable thing.

Concatenate a list of buffers into a single unified buffer. This shares the data and it works when the number of buffers is relatively small. The failure case for this is a large number of small buffers where it will be faster to simply copy all the data into a new buffer.

Create a new reader that only returns the item for the provided indexes.

Mutably copy values from a src container into a destination container. Returns the destination container.

Copy raw data into a buffer. Data may be a sequence of numbers or a sequence of containers. Data will be coalesced into the buffer. Returns a tuple of: buffer final-offset.

Return this object's actual datatype. This is not the same as the DEPRECATED get-datatype. That function maps to elemwise-datatype. This maps to the parameterization of the object, so for instance a list of ints might be:

  {:container-type :list :elemwise-datatype :int32}

Defaults to this object's elemwise-datatype.

Return the long number of elements in the object.

Create a new thing by doing a checked cast from the old values to the new values upon at read time. Return value will report new-dtype as it's elemwise-datatype.

Return the datatype one would expect when iterating through a container of this type. For scalars, return your elemental datatype.

Elemwise map:

1. If input are all scalars, results in a scalar.
2. If any inputs are iterables, results in an iterable.
3. Either a reader or a tensor is returned. All input shapes have to match.

res-dtype is nil it is deduced from unifying the argument datatypes meaning map-fn, unless it is typed, is assumed to keep data in the same numeric space as the input.

If input is already a native buffer and has the same datatype as output, return input. Else copy input into output and return output.

Ensure item is randomly addressable. This may copy the data into a randomly accessible container.

Ensure this is an efficiently serializeable datatype object. For nippy support across buffers and tensors require tech.v3.datatype.nippy.

Legacy method, returns elemwise-datatype

Get a value from an object via conversion to a reader.

Create a new Buffer implementation that indexes into a previous Buffer implementation via the provided indexes.

Return true if this is either a instance of Iterable or an instance of java.util.streams.Stream.

Make a container of a given datatype. Options are container specific and in general unused. Values will be copied into given container using the most efficient pathway possible.

Make an instance of a tech.v3.datatype.Buffer. These have typed add* methods, implement tech.v3.datatype.Buffer, and a guaranteed in-place transformation to a concrete buffer (defaults to an array buffer).

If an integer is passed in an empty list that has preallocated storage is returned. If a sequence of data is passed in then a non-empty list that contains just those elements is returned.

Example:

user> (require '[tech.v3.datatype :as dt])
nil
user> (dt/make-list :float32 10)
#list<float32>[0]
[]
user> (dt/make-list :float32 (range 10))
#list<float32>[10]
[0.000, 1.000, 2.000, 3.000, 4.000, 5.000, 6.000, 7.000, 8.000, 9.000]

Make a reader. Datatype must be a compile time visible object. read-op has 'idx' in scope which is the index to read from. Returns a reader of the appropriate type for the passed in datatype. Results are unchecked casted to the appropriate datatype. It is up to you to ensure this is the result you want or throw an exception.

This function creates a compile-time object that can allow maximum performance. For most use cases this is probably overkill and 'emap' is a more ideal pathway that matches Clojure constructs a bit closer.

reader-datatype must be a compile time constant but advertised datatype need not be.

user> (dtype/make-reader :float32 5 idx) 0.0 1.0 2.0 3.0 4.0 user> (dtype/make-reader :boolean 5 idx) true true true true true user> (dtype/make-reader :boolean 5 (== idx 0)) true false false false false user> (dtype/make-reader :float32 5 (* idx 2)) 0.0 2.0 4.0 6.0 8.0

Malloc memory. If a desired buffer type is needed follow up with set-native-datatype.

• n-bytes - the number of bytes to malloc
• opts - Either a keyword datatype or an options map.

Options:

• :resource-type - defaults to :gc - maps to :track-type in tech.v3.resource/track but can also be set to nil in which case the data is not tracked this library will not clean it up.
• :datatype - Datatype - defaults to :int8. Datatype byte width must evenly divide n-bytes.
• :uninitialized? - do not initialize to zero. Use for perf in very very rare cases.
• :endianness - Either :little-endian or :big-endian - defaults to platform.
• :log-level - one of #{:debug :trace :info :warn :error :fatal} or nil if no logging is desired. When enabled allocations and frees will be logged in the same manner as tech.jna.

Make an list with preallocated storage. This function exists to cause a compilation error if older versions of dtype-next are included and is the equivalent (in latest versions of dtype-next) to (make-list datatype n-elems).

Returns true if this argument type is convertible to a reader.

True if this item is convertible to a read-only or read-write Buffer interface.

Reverse an sequence, range or reader.

• If range, returns a new range.
• If sequence, uses clojure.core/reverse
• If reader, returns a new reader that performs an in-place reverse.

Set a contiguous region of a buffer to a constant value

Legacy method. Performs elemwise-cast.

Set the datatype of a native buffer. n-elems will be recalculated.

Set a value on an object via conversion to a writer. set-value can also take a tuple in which case a select operation will be done on the tensor and value applied to the result. See also set-constant!.

Example:

tech.v3.tensor.integration-test> (def test-tens (dtt/->tensor (->> (range 27)
                                     (partition 3)
                                     (partition 3))
                                {:datatype :float64}))

#'tech.v3.tensor.integration-test/test-tens
tech.v3.tensor.integration-test> test-tens
#tech.v3.tensor<float64>[3 3 3]
[[[0.000 1.000 2.000]
  [3.000 4.000 5.000]
  [6.000 7.000 8.000]]
 [[9.000 10.00 11.00]
  [12.00 13.00 14.00]
  [15.00 16.00 17.00]]
 [[18.00 19.00 20.00]
  [21.00 22.00 23.00]
  [24.00 25.00 26.00]]]
tech.v3.tensor.integration-test> (dtype/set-value! (dtype/clone test-tens) [:all :all (range 2)] 0)
#tech.v3.tensor<float64>[3 3 3]
[[[0.000 0.000 2.000]
  [0.000 0.000 5.000]
  [0.000 0.000 8.000]]
 [[0.000 0.000 11.00]
  [0.000 0.000 14.00]
  [0.000 0.000 17.00]]
 [[0.000 0.000 20.00]
  [0.000 0.000 23.00]
  [0.000 0.000 26.00]]]
tech.v3.tensor.integration-test> (def sv-tens (dtt/reshape (double-array [1 2 3]) [3 1]))
#'tech.v3.tensor.integration-test/sv-tens
tech.v3.tensor.integration-test> (dtype/set-value! (dtype/clone test-tens) [:all :all (range 2)] sv-tens)
#tech.v3.tensor<float64>[3 3 3]
[[[1.000 1.000 2.000]
  [2.000 2.000 5.000]
  [3.000 3.000 8.000]]
 [[1.000 1.000 11.00]
  [2.000 2.000 14.00]
  [3.000 3.000 17.00]]
 [[1.000 1.000 20.00]
  [2.000 2.000 23.00]
  [3.000 3.000 26.00]]]

Return a persistent vector of the shape of the object. Dimensions are returned from least-rapidly-changing to most-rapidly-changing.

Create a sub-buffer (view) of the given buffer.

Perform an unchecked cast of a value to specific datatype.

Perform a vectorized dispatch meaning return a different object depending on the argument type of arg1. If arg1 is scalar, use scalar-fn. If arg1 is an iterable, use iterable-fn. Finally if arg1 is a reader, use reader-fn. The space the operation is to be performed in can be loosely set with :operation-space in the options. The actual space the operation will be performed in (and the return type of the system) is a combination of :operation-space if it exists and the elemwise datatype of arg1. See tech.v3.datatype.casting/widest-datatype for more information.

Perform a vectorized dispatch meaning return a different object depending on the argument types of arg1 and arg2. If arg1 and arg2 are both scalar, use scalar-fn. If neither argument is a reader, use iterable-fn. Finally if both arguments are readers, use reader-fn. The space the operation is to be performed in can be loosely set with :operation-space in the options. The actual space the operation will be performed in (and the return type of the system) is a combination of :operation-space if it exists and the elemwise datatypes of arg1 and arg2. See tech.v3.datatype.casting/widest-datatype for more information.

True if this item is convertible to a Buffer interface that supports writing.