Package tech.v3

Class Tensor

  • public class Tensor
extends java.lang.Object

    The Tensor api exposes a simple and efficient NDBuffer implementation with of zero-copy access to many numeric systems such as Julia, Numpy, and Clojure's Neanderthal linear algebraic system. The primary interface for this library is tech.v3.datatype.NDBuffer.

    Given a tech.v3.DType.toBuffer-capable object such as a primitive array or a contiguous block of native memory, a 'reshape' call returns an NDBuffer or you can copy arbitrary data into a container with 'makeTensor'.

    You can mutate data with the mset api function or directly via the NDBuffer interface methods.

    Any time a tensor is linearized it is read row-major. So if you call any base DType functions on it such as 'toDoubleArray' you will get the data in row-major form. If this tensors is already using a double array buffer such as if you just reshaped a double array it will simply return its buffer.

    Once you have a tensor then there are a few methods of interacting with it. All of these methods create views:

    • select - select a region or reorder within a dimension.
    • transpose - reorder dimensions themselves.
    • reshape - reshape the source buffer into a different NDBuffer.
    • broadcast - enlarge the NDBuffer by duplicating dimensions.
    • slice(Right) - slice a fixed number of dimensions from the left or right of the tensor returning a list of sub-tensors. Slicing all dimensions is equivalent to calling Dtype.toBuffer() meaning you will just get a flat Buffer implementation in row-major form.

    The C ABI of the tensor system is a map with at least specific keys in it. Necessary keys are clojure keywords:

    • :ptr - Long integer ptr to data.
    • :elemwise-datatype - One of the numeric datatypes.
    • :endianness - One of ':little-endian' or ':big-endian'.
    • :shape - Integer or Long array of dimensions.
    • :strides - Integer or Long array of byte strides.

    Example - Note there are extra keys in the descriptor map. This is fine; extra keys are ignored:

      try (AutoCloseable ac = stackResourceContext()) {
    Map bufDesc = ensureNDBufferDescriptor(reshape(toDoubleArray(range(9)), vector(3,3)));
    System.out.println(bufDesc.toString());
    System.out.println(NDBufferDescriptorToTensor(bufDesc).toString());
  } catch(Exception e){
    System.out.println(e.toString());
    e.printStackTrace(System.out);
  }
 //Outputs-
 {:ptr 140054473349344, :datatype :tensor, :elemwise-datatype :float64, :endianness :little-endian, :shape [3 3], :strides [24 8], :native-buffer #native-buffer@0x00007F60F92218E0[9]
 [0.000, 1.000, 2.000, 3.000, 4.000, 5.000, 6.000, 7.000, 8.000]}
 #tech.v3.tensor[3 3]
 [[0.000 1.000 2.000]
 [3.000 4.000 5.000]
 [6.000 7.000 8.000]]

    The entire resource descriptor is as the native buffer's gc object so anything else referenced in the metadata will survive as long as the native buffer is referenced.

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static tech.v3.datatype.NDBuffer asTensor​(java.lang.Object data)
      Attempt an in-place conversion of the data such as a numpy array into a tensor.
      static tech.v3.datatype.NDBuffer broadcast​(java.lang.Object src, java.lang.Object newShape)
      Enlarge the tensor by duplicating dimenions.
      static java.util.List columns​(tech.v3.datatype.NDBuffer src)
      Return the columns.
      static tech.v3.datatype.NDBuffer computeTensor​(java.lang.Object datatype, java.lang.Object shape, clojure.lang.IFn indexFn)
      Make a virtualized tensor that will on-demand call indexFn for each element.
      static void enableNeanderthal()
      Attempt to load neanderthal bindings.
      static java.util.Map ensureNDBufferDescriptor​(java.lang.Object src)
      Get an NDBufferDescription from a tensor.
      static tech.v3.datatype.NDBuffer makeTensor​(java.lang.Object data, java.lang.Object shape, java.lang.Object datatype)
      Make an tensor by copying data into a tensor of a given shape and datatype.
      static tech.v3.datatype.NDBuffer makeTensor​(java.lang.Object data, java.lang.Object shape, java.lang.Object datatype, java.lang.Object containerType)
      Make an tensor by copying data into a tensor of a given shape, datatype and container type.
      static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens, long dim)
      Return the value of the tensor at the leftmost dimension at location dim.
      static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens, long dim1, long dim2)
      Return the value of the tensor at the provided dimensions.
      static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens, long dim1, long dim2, long dim3)
      Return the value of the tensor at the provided dimensions.
      static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens, long dim1, long dim2, long dim3, long dim4)
      Return the value of the tensor at the provided dimensions.
      static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens, long dim1, long dim2, long dim3, long dim4, java.lang.Object val)
      Set the sub-tensor at [dim1,dim2,dim3,dim4] to value val.
      static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens, long dim1, long dim2, long dim3, java.lang.Object val)
      Set the sub-tensor at [dim1,dim2,dim3] to value val.
      static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens, long dim1, long dim2, java.lang.Object val)
      Set the sub-tensor at [dim1,dim2] to value val.
      static java.lang.Object mset​(tech.v3.datatype.NDBuffer tens, long dim, java.lang.Object val)
      Set the sub-tensor at dim to value val.
      static java.lang.Object mset​(tech.v3.datatype.NDBuffer tens, java.lang.Object val)
      Set the tensor to a value.
      static tech.v3.datatype.NDBuffer NDBufferDescriptorToTensor​(java.util.Map ndBufferDesc)
      Given an ndbufferdescriptor create a tensor.
      static tech.v3.datatype.NDBuffer reshape​(java.lang.Object src, java.lang.Object newShape)
      Reshape a tensor by first getting it's row-major internal representation and then applying a new shape to it.
      static java.util.List rows​(tech.v3.datatype.NDBuffer src)
      Return the rows.
      static tech.v3.datatype.NDBuffer select​(tech.v3.datatype.NDBuffer src, java.lang.Object... dims)
      Select a subrect and potentially reindex a tensor.
      static java.util.List slice​(tech.v3.datatype.NDBuffer src, long ndims)
      Slice a tensor returning a list of sub-tensors.
      static java.util.List sliceRight​(tech.v3.datatype.NDBuffer src, long ndims)
      Slice a tensor logically from the right.
      static java.lang.Object tensorToNeanderthalMatrix​(java.lang.Object tens)
      Convert a tensor to a column major neanderthal matrix of the same datatype as the tensor.
      static java.lang.Object tensorToNeanderthalMatrix​(java.lang.Object tens, clojure.lang.Keyword layout, clojure.lang.Keyword datatype)
      Convert a tensor to a neanderthal matrix.
      static tech.v3.datatype.NDBuffer transpose​(tech.v3.datatype.NDBuffer src, java.lang.Object dimIndexes)
      Transpose an NDBuffer by reordering its dimensions.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • makeTensor

        public static tech.v3.datatype.NDBuffer makeTensor​(java.lang.Object data,
                                                   java.lang.Object shape,
                                                   java.lang.Object datatype)
        Make an tensor by copying data into a tensor of a given shape and datatype.

      • makeTensor

        public static tech.v3.datatype.NDBuffer makeTensor​(java.lang.Object data,
                                                   java.lang.Object shape,
                                                   java.lang.Object datatype,
                                                   java.lang.Object containerType)
        Make an tensor by copying data into a tensor of a given shape, datatype and container type.

      • computeTensor

        public static tech.v3.datatype.NDBuffer computeTensor​(java.lang.Object datatype,
                                                      java.lang.Object shape,
                                                      clojure.lang.IFn indexFn)
        Make a virtualized tensor that will on-demand call indexFn for each element. A 2D tensor will call the 2-arity invoke overload with Long arguments. The result will be unceremoniously forced into the datatype indicated by datatype.

      • asTensor

        public static tech.v3.datatype.NDBuffer asTensor​(java.lang.Object data)
        Attempt an in-place conversion of the data such as a numpy array into a tensor.

      • mget

        public static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens,
                                    long dim)
        Return the value of the tensor at the leftmost dimension at location dim.

      • mget

        public static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens,
                                    long dim1,
                                    long dim2)
        Return the value of the tensor at the provided dimensions.

      • mget

        public static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens,
                                    long dim1,
                                    long dim2,
                                    long dim3)
        Return the value of the tensor at the provided dimensions.

      • mget

        public static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens,
                                    long dim1,
                                    long dim2,
                                    long dim3,
                                    long dim4)
        Return the value of the tensor at the provided dimensions.

      • mset

        public static java.lang.Object mset​(tech.v3.datatype.NDBuffer tens,
                                    java.lang.Object val)
        Set the tensor to a value. If this value is a constant, the tensor will be constant value. Else val must have same dimensions as tens.

      • mset

        public static java.lang.Object mset​(tech.v3.datatype.NDBuffer tens,
                                    long dim,
                                    java.lang.Object val)
        Set the sub-tensor at dim to value val. val must either be constant or have the same dimensions as the sub-tensor.

      • mget

        public static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens,
                                    long dim1,
                                    long dim2,
                                    java.lang.Object val)
        Set the sub-tensor at [dim1,dim2] to value val. val must either be constant or have the same dimensions as the sub-tensor.

      • mget

        public static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens,
                                    long dim1,
                                    long dim2,
                                    long dim3,
                                    java.lang.Object val)
        Set the sub-tensor at [dim1,dim2,dim3] to value val. val must either be constant or have the same dimensions as the sub-tensor.

      • mget

        public static java.lang.Object mget​(tech.v3.datatype.NDBuffer tens,
                                    long dim1,
                                    long dim2,
                                    long dim3,
                                    long dim4,
                                    java.lang.Object val)
        Set the sub-tensor at [dim1,dim2,dim3,dim4] to value val. val must either be constant or have the same dimensions as the sub-tensor.

      • broadcast

        public static tech.v3.datatype.NDBuffer broadcast​(java.lang.Object src,
                                                  java.lang.Object newShape)
        Enlarge the tensor by duplicating dimenions. newShape's dimensions must all be identical or evenly divisible by the shape of src.

      • reshape

        public static tech.v3.datatype.NDBuffer reshape​(java.lang.Object src,
                                                java.lang.Object newShape)
        Reshape a tensor by first getting it's row-major internal representation and then applying a new shape to it. New shape need to use every element in src.

      • select

        public static tech.v3.datatype.NDBuffer select​(tech.v3.datatype.NDBuffer src,
                                               java.lang.Object... dims)
        Select a subrect and potentially reindex a tensor. Starting from the left if dim is a number then that exact position is select. If dim is convertible to a buffer then that dimension is reindexed by the given elements. The keyword :all may be used to indicate use entire dimension. Any dimensions that aren't included in the selection are left unchanged. See examples in the Clojure documentation.

      • transpose

        public static tech.v3.datatype.NDBuffer transpose​(tech.v3.datatype.NDBuffer src,
                                                  java.lang.Object dimIndexes)
        Transpose an NDBuffer by reordering its dimensions. For example given an image of shape [Y X C] we can make a planar representation by moving the the channels first corresponds to reindexing dimensions [2 0 1]. See examples in the Clojure documentation.

      • slice

        public static java.util.List slice​(tech.v3.datatype.NDBuffer src,
                                   long ndims)
        Slice a tensor returning a list of sub-tensors. Slicing an image of shape [Y X C] by 1 returns a list of [X C] tensors. Slicing by 2 returns a list of [C] tensors and slicing by 3 returns the tensor as a linear buffer.

      • sliceRight

        public static java.util.List sliceRight​(tech.v3.datatype.NDBuffer src,
                                        long ndims)
        Slice a tensor logically from the right. Given an RGB image of shape [Y X C] slicing right by 1 returns a list of length 3 of [Y X] tensors - the red plane followed by the green plane and finally the blue plane.

      • rows

        public static java.util.List rows​(tech.v3.datatype.NDBuffer src)
        Return the rows. Equivalent to slice(src, 1).

      • columns

        public static java.util.List columns​(tech.v3.datatype.NDBuffer src)
        Return the columns. Equivalent to sliceRight(src, 1).

      • ensureNDBufferDescriptor

        public static java.util.Map ensureNDBufferDescriptor​(java.lang.Object src)
        Get an NDBufferDescription from a tensor. In the case where the tensors underlying representation is on the jvm-heap this will copy the data into a new native-heap representation.

      • NDBufferDescriptorToTensor

        public static tech.v3.datatype.NDBuffer NDBufferDescriptorToTensor​(java.util.Map ndBufferDesc)
        Given an ndbufferdescriptor create a tensor. The tensor will reference the buffer descriptor in its metadata.

      • enableNeanderthal

        public static void enableNeanderthal()
        Attempt to load neanderthal bindings. Throws exception upon failure.

      • tensorToNeanderthalMatrix

        public static java.lang.Object tensorToNeanderthalMatrix​(java.lang.Object tens,
                                                         clojure.lang.Keyword layout,
                                                         clojure.lang.Keyword datatype)
        Convert a tensor to a neanderthal matrix.
        Parameters:
        layout - Either `:column` or `:row`.
        datatype - Either `:float64` or `:float32`.

      • tensorToNeanderthalMatrix

        public static java.lang.Object tensorToNeanderthalMatrix​(java.lang.Object tens)
        Convert a tensor to a column major neanderthal matrix of the same datatype as the tensor. Supports either `:float32` or `:float64` tensors.