tech.v3.datatype.functional

Arithmetic and statistical operations based on the Buffer interface. These operators and functions all implement vectorized interfaces so passing in something convertible to a reader will return a reader. Arithmetic operations are done lazily. These functions generally incur a large dispatch cost so for example each call to '+' checks all the arguments to decide if it should dispatch to an iterable implementation or to a reader implementation. For tight loops or operations like map and filter, using the specific operators will result in far faster code than using the '+' function itself.

*

(* x y)(* x y & args)

+

(+ x)(+ x y)(+ x y & args)

-

(- x)(- x y)(- x y & args)

/

(/ x)(/ x y)(/ x y & args)

<

(< lhs mid rhs)(< lhs rhs)

<=

(<= lhs mid rhs)(<= lhs rhs)

>

(> lhs mid rhs)(> lhs rhs)

>=

(>= lhs mid rhs)(>= lhs rhs)

abs

(abs x options)(abs x)

acos

(acos x options)(acos x)

and

(and lhs rhs)

asin

(asin x options)(asin x)

atan

(atan x options)(atan x)

atan2

(atan2 x y)(atan2 x y & args)

bit-and

(bit-and x y)(bit-and x y & args)

bit-and-not

(bit-and-not x y)(bit-and-not x y & args)

bit-clear

(bit-clear x y)(bit-clear x y & args)

bit-flip

(bit-flip x y)(bit-flip x y & args)

bit-not

(bit-not x options)(bit-not x)

bit-or

(bit-or x y)(bit-or x y & args)

bit-set

(bit-set x y)(bit-set x y & args)

bit-shift-left

(bit-shift-left x y)(bit-shift-left x y & args)

bit-shift-right

(bit-shift-right x y)(bit-shift-right x y & args)

bit-test

(bit-test lhs rhs)

bit-xor

(bit-xor x y)(bit-xor x y & args)

bool-reader->indexes

(bool-reader->indexes options bool-item)(bool-reader->indexes bool-item)

Given a reader, produce a filtered list of indexes filtering out 'false' values.

cbrt

(cbrt x options)(cbrt x)

ceil

(ceil x options)(ceil x)

cos

(cos x options)(cos x)

cosh

(cosh x options)(cosh x)

cummax

(cummax options data)(cummax data)

Cumulative running max; returns result in double space.

Options:

  • :nan-strategy - one of :keep, :remove, :exception. Defaults to :remove.

cummin

(cummin options data)(cummin data)

Cumulative running min; returns result in double space.

Options:

  • :nan-strategy - one of :keep, :remove, :exception. Defaults to :remove.

cumop

(cumop options op data)

cumprod

(cumprod options data)(cumprod data)

Cumulative running product; returns result in double space.

Options:

  • :nan-strategy - one of :keep, :remove, :exception. Defaults to :remove.

cumsum

(cumsum options data)(cumsum data)

Cumulative running summation; returns result in double space.

Options:

  • :nan-strategy - one of :keep, :remove, :exception. Defaults to :remove.

descriptive-statistics

(descriptive-statistics stats-names stats-data options src-rdr)(descriptive-statistics stats-names options rdr)(descriptive-statistics stats-names rdr)(descriptive-statistics rdr)

Calculate a set of descriptive statistics on a single reader.

Available stats: #{:min :quartile-1 :sum :mean :mode :median :quartile-3 :max :variance :standard-deviation :skew :n-elems :kurtosis}

options

  • :nan-strategy - defaults to :remove, one of :keep :remove :exception. The fastest option is :keep but this may result in your results having NaN's in them. You can also pass in a double predicate to filter custom double values.

distance

(distance lhs rhs)

distance-squared

(distance-squared lhs rhs)

dot-product

(dot-product lhs rhs)

eq

(eq lhs rhs)

equals

(equals lhs rhs & args)

even?

(even? arg options)(even? arg)

exp

(exp x options)(exp x)

expm1

(expm1 x options)(expm1 x)

fill-range

(fill-range numeric-data max-span)

Given a reader of numeric data and a max span amount, produce a new reader where the difference between any two consecutive elements is less than or equal to the max span amount. Also return a bitmap of the added indexes. Uses linear interpolation to fill in areas, operates in double space. Returns {:result :missing}

finite?

(finite? arg options)(finite? arg)

fixed-rolling-window

(fixed-rolling-window item window-size window-fn options)(fixed-rolling-window item window-size window-fn)

Return a lazily evaluated rolling window of window-fn applied to each window. The iterable or sequence is padded such that there are the same number of values in the result as in the input with repeated elements padding the beginning and end of the original sequence. If input is an iterator, output is an lazy sequence. If input is a reader, output is a reader.

:Options

  • :relative-window-position - Defaults to :center - controls the window's relative positioning in the sequence.
  • :edge-mode - Defaults to :clamp - either :zero in which case window values off the edge are zero for numeric types or nil for object types or :clamp - in which case window values off the edge of the data are bound to the first or last values respectively.

Example (all results are same length):

user> (require '[tech.v3.datatype :as dtype])
nil
user> (require '[tech.v3.datatype.rolling :as rolling])
nil
user> (require '[tech.v3.datatype.functional :as dfn])
nil
  user> (rolling/fixed-rolling-window (range 20) 5 dfn/sum {:relative-window-position :left})
[0 1 3 6 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85]
user> (rolling/fixed-rolling-window (range 20) 5 dfn/sum {:relative-window-position :center})
[3 6 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 89 92]
user> (rolling/fixed-rolling-window (range 20) 5 dfn/sum {:relative-window-position :right})
[10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 89 92 94 95]
user>

floor

(floor x options)(floor x)

get-significand

(get-significand x options)(get-significand x)

hypot

(hypot x y)(hypot x y & args)

identity

(identity x options)(identity x)

ieee-remainder

(ieee-remainder x y)(ieee-remainder x y & args)

infinite?

(infinite? arg options)(infinite? arg)

kendalls-correlation

(kendalls-correlation options lhs rhs)(kendalls-correlation lhs rhs)

kurtosis

(kurtosis data options)(kurtosis data)

linear-regressor

(linear-regressor x y)

Create a simple linear regressor. Returns a function that given a (double) 'x' predicts a (double) 'y'. The function has metadata that contains the regressor and some regressor info, notably slope and intercept.

Example:

tech.v3.datatype.functional> (def regressor (linear-regressor [1 2 3] [4 5 6]))
#'tech.v3.datatype.functional/regressor
tech.v3.datatype.functional> (regressor 1)
4.0
tech.v3.datatype.functional> (regressor 2)
5.0
tech.v3.datatype.functional> (meta regressor)
{:regressor
  #object[org.apache.commons.math3.stat.regression.SimpleRegression 0x52091e82 "org.apache.commons.math3.stat.regression.SimpleRegression@52091e82"],
 :intercept 3.0,
 :slope 1.0,
 :mean-squared-error 0.0}

log

(log x options)(log x)

log10

(log10 x options)(log10 x)

log1p

(log1p x options)(log1p x)

logistic

(logistic x options)(logistic x)

magnitude

(magnitude item _options)(magnitude item)

magnitude-squared

(magnitude-squared data)

mathematical-integer?

(mathematical-integer? arg options)(mathematical-integer? arg)

max

(max x)(max x y)(max x y & args)

mean

(mean data options)(mean data)

double mean of data

mean-fast

(mean-fast data)

Take the mean of the data. This operation doesn't know anything about nan hence it is a bit faster than the base mean fn.

median

(median data options)(median data)

min

(min x)(min x y)(min x y & args)

mode

(mode data)

Return the value of the most common occurance in the data.

nan?

(nan? arg options)(nan? arg)

neg?

(neg? arg options)(neg? arg)

next-down

(next-down x options)(next-down x)

next-up

(next-up x options)(next-up x)

normalize

(normalize item)

not

(not arg options)(not arg)

not-eq

(not-eq lhs rhs)

odd?

(odd? arg options)(odd? arg)

or

(or lhs rhs)

pearsons-correlation

(pearsons-correlation options lhs rhs)(pearsons-correlation lhs rhs)

percentiles

(percentiles percentages options data)(percentiles percentages data)

Create a reader of percentile values, one for each percentage passed in. Estimation types are in the set of #{:r1,r2...legacy} and are described here: https://commons.apache.org/proper/commons-math/javadocs/api-3.3/index.html.

nan-strategy can be one of :keep :remove :exception and defaults to :exception.

pos?

(pos? arg options)(pos? arg)

pow

(pow x y)(pow x y & args)

quartile-1

(quartile-1 data options)(quartile-1 data)

quartile-3

(quartile-3 data options)(quartile-3 data)

quartile-outlier-fn

(quartile-outlier-fn item & args)

Create a function that, given floating point data, will return true or false if that data is an outlier. Default range mult is 1.5: (or (< val (- q1 (* range-mult iqr))) (> val (+ q3 (* range-mult iqr)))

quartiles

(quartiles item)(quartiles options item)

return min, 25 50 75 max of item

quot

(quot x y)(quot x y & args)

reduce-*

(reduce-* rdr)

reduce-+

(reduce-+ rdr)

reduce-max

(reduce-max rdr)

reduce-min

(reduce-min rdr)

rem

(rem x y)(rem x y & args)

rint

(rint x options)(rint x)

round

(round arg options)(round arg)

Vectorized implementation of Math/round. Operates in double space but returns a long or long reader.

shift

(shift rdr n)

Shift by n and fill in with the first element for n>0 or last element for n<0.

Examples:

user> (dfn/shift (range 10) 2)
[0 0 0 1 2 3 4 5 6 7]
user> (dfn/shift (range 10) -2)
[2 3 4 5 6 7 8 9 9 9]

signum

(signum x options)(signum x)

sin

(sin x options)(sin x)

sinh

(sinh x options)(sinh x)

skew

(skew data options)(skew data)

spearmans-correlation

(spearmans-correlation options lhs rhs)(spearmans-correlation lhs rhs)

sq

(sq x options)(sq x)

sqrt

(sqrt x options)(sqrt x)

standard-deviation

(standard-deviation data options)(standard-deviation data)

sum

(sum data options)(sum data)

Double sum of data using Kahan compensated summation.

sum-fast

(sum-fast data)

Find the sum of the data. This operation is neither nan-aware nor does it implement kahans compensation although via parallelization it implements pairwise summation compensation. For a more but slightly slower but far more correct sum operator, use sum.

tan

(tan x options)(tan x)

tanh

(tanh x options)(tanh x)

to-degrees

(to-degrees x options)(to-degrees x)

to-radians

(to-radians x options)(to-radians x)

ulp

(ulp x options)(ulp x)

unsigned-bit-shift-right

(unsigned-bit-shift-right x y)(unsigned-bit-shift-right x y & args)

variance

(variance data options)(variance data)

zero?

(zero? arg options)(zero? arg)