Operators

This chapter will go through the operators and Ext module. The operators in Owl are implemented in the functors defined in the Owl_operator module. These operators are categorised into Basic, Extend, Matrix, and Ndarray four module type signatures, because some operations are only meaningful for certain data structures. E.g., matrix multiplication *@ is only defined in Matrix signature.

As long as a module implements the functions defined in the module signature, you can use these functors to generate corresponding operators. However, you do not need to work with functors directly in Owl since I have done the generation part for you already.

Basic Operators

The operators have been included in each Ndarray and Matrix module. The following table summarises the operators currently implemented in Owl. In the table, both x and y represent either a matrix or an ndarray while a represents a scalar value.

Operator Example Operation Dense/Sparse Ndarray/Matrix
+ x + y element-wise add both both
- x - y element-wise sub both both
* x * y element-wise mul both both
/ x / y element-wise div both both
+$ x +$ a add scalar both both
-$ x -$ a sub scalar both both
*$ x *$ a mul scalar both both
/$ x /$ a div scalar both both
$+ a $+ x scalar add both both
$- a $- x scalar sub both both
$* a $* x scalar mul both both
$/ a $/ x scalar div both both
= x = y comparison both both
!= x != y comparison both both
<> x <> y same as != both both
> x > y comparison both both
< x < y comparison both both
>= x >= y comparison both both
<= x <= y comparison both both
=. x =. y element-wise cmp Dense both
!=. x !=. y element-wise cmp Dense both
<>. x <>. y same as !=. Dense both
>. x >. y element-wise cmp Dense both
<. x <. y element-wise cmp Dense both
>=. x >=. y element-wise cmp Dense both
<=. x <=. y element-wise cmp Dense both
=$ x =$ y comp to scalar Dense both
!=$ x !=$ y comp to scalar Dense both
<>$ x <>$ y same as != Dense both
>$ x >$ y compare to scalar Dense both
<$ x <$ y compare to scalar Dense both
>=$ x >=$ y compare to scalar Dense both
<=$ x <=$ y compare to scalar Dense both
=.$ x =.$ y element-wise cmp Dense both
!=.$ x !=.$ y element-wise cmp Dense both
<>.$ x <>.$ y same as !=.$ Dense both
>.$ x >.$ y element-wise cmp Dense both
<.$ x <.$ y element-wise cmp Dense both
>=.$ x >=.$ y element-wise cmp Dense both
<=.$ x <=.$ y element-wise cmp Dense both
=~ x =~ y approx = Dense both
=~$ x =~$ y approx =$ Dense both
=~. x =~. y approx =. Dense both
=~.$ x =~.$ y approx =.$ Dense both
% x % y mod divide Dense both
%$ x %$ a mod divide scalar Dense both
** x ** y power function Dense both
*@ x *@ y matrix multiply both Matrix
/@ x /@ y solve linear system both Matrix
**@ x **@ a matrix power both Matrix
min2 min2 x y element-wise min both both
max2 max2 x y element-wise max both both
@= x @= y concatenate vertically Dense both
@|| x @|| y concatenate horizontally Dense both

There are a list of things worth your attention as below.

  • * is for element-wise multiplication; *@ is for matrix multiplication. You can easily understand the reason if you read the source code of Algodiff module. Using * for element-wise multiplication (for matrices) leads to the consistent implementation of algorithmic differentiation.
  • +$ has its corresponding operator $+ if we flip the order of parameters. However, be very careful about the operator precedence since OCaml determines the precedence based on the first character of an infix. +$ preserves the precedence whereas $+ does not. Therefore, I do not recommend using $+. If you do use it, please use parentheses to explicitly specify the precedence. The same argument also applies to $-, $*, and $/.
  • For comparison operators, e.g. both = and =. compare all the elements in two variables x and y. The difference is that = returns a boolean value whereas =. returns a matrix or ndarray of the same shape and same type as x and y. In the returned result, the value in a given position is 1 if the values of the corresponding position in x and y satisfy the predicate, otherwise it is 0.
  • For the comparison operators ended with $, they are used to compare a matrix/ndarray to a scalar value.

Operators are easy to use, here are some examples.

let x = Mat.uniform 5 5;;
let y = Mat.uniform 5 5;;

Mat.(x + y);;
Mat.(x * y);;
Mat.(x ** y);;
Mat.(x *@ y);;

...

(* please compare the returns of the following two examples *)
Mat.(x > y);;
Mat.(x >. y);;

Extending indexing and slicing operators are not included in the table above, but you can find the detailed explanation in Slicing Chapter.

Extension Module

As you can see, the operators above do not allow interoperation on different number types (which may not be bad thing in many cases actually). E.g., you cannot add a float32 matrix to float64 matrix unless you explicitly call the cast functions in Generic module {read this}.

Owl.Ext module is specifically designed for this purpose, to make prototyping faster and easier. Once you open the module, Ext immediately provides a set of operators to allow you to interoperate on different number types, as below. It automatically casts types for you if necessary.

Operator Example Operation
+ x + y add
- x - y sub
* x * y mul
/ x / y div
= x = y comparison, return bool
!= x != y comparison, return bool
<> x <> y same as !=
> x > y comparison, return bool
< x < y comparison, return bool
>= x >= y comparison, return bool
<= x <= y comparison, return bool
=. x =. y element_wise comparison
!=. x !=. y element_wise comparison
<>. x <>. y same as !=.
>. x >. y element_wise comparison
<. x <. y element_wise comparison
>=. x >=. y element_wise comparison
<=. x <=. y element_wise comparison
% x % y element_wise mod divide
** x ** y power function
*@ x *@ y matrix multiply
min2 min2 x y element-wise min
max2 max2 x y element-wise max

You may have noticed, the operators ended with $ (e.g., +$, -$ …) disappeared from the table, which is simply because we can add/sub/mul/div a scalar with a matrix directly and we do not need these operators any more. Similar for comparison operators, because we can use the same > operator to compare a matrix to another matrix, or compare a matrix to a scalar, we do not need >$ any longer. Allowing interoperation makes the operator table much shorter.

Currently, the operators in Ext only support interoperation on dense structures. Besides binary operators, Ext also implements most of the common math functions which can be applied to float numbers, complex numbers, matrices, and ndarray. These functions are:

im; re; conj, abs, abs2, neg, reci, signum, sqr, sqrt, cbrt, exp, exp2, expm1, log, log10, log2, log1p, sin, cos, tan, asin, acos, atan, sinh, cosh, tanh, asinh, acosh, atanh, floor, ceil, round, trunc, erf, erfc, logistic, relu, softplus, softsign, softmax, sigmoid, log_sum_exp, l1norm, l2norm, l2norm_sqr, inv, trace, sum, prod, min, max, minmax, min_i, max_i, minmax_i.

Note that Ext contains its own Ext.Dense module which further contains the following submodules.

  • Ext.Dense.Ndarray.S
  • Ext.Dense.Ndarray.D
  • Ext.Dense.Ndarray.C
  • Ext.Dense.Ndarray.Z
  • Ext.Dense.Matrix.S
  • Ext.Dense.Matrix.D
  • Ext.Dense.Matrix.C
  • Ext.Dense.Matrix.Z

These modules are simply the wrappers of the original modules in Owl.Dense module so they provide most of the APIs already implemented. The extra thing these wrapper modules does is to pack and unpack the raw number types for you automatically. However, you can certainly use the raw data types then use the constructors defined in Owl_ext_types to wrap them up by yourself. The constructors are defined as below.

type ext_typ =
  F   of float
  C   of Complex.t
  DMS of dms
  DMD of dmd
  DMC of dmc
  DMZ of dmz
  DAS of das
  DAD of dad
  DAC of dac
  DAZ of daz
  SMS of sms
  SMD of smd
  SMC of sms
  SMZ of smd
  SAS of sas
  SAD of sad
  SAC of sac
  SAZ of saz

There are also corresponding packing and unpacking functions you can use, please read owl_ext_types.ml for more details.

Let’s see some examples to understand how convenient it is to use Ext module.

open Owl.Ext;;

let x = Dense.Matrix.S.uniform 5 5;;
let y = Dense.Matrix.C.uniform 5 5;;
let z = Dense.Matrix.D.uniform 5 5;;

x + F 5.;;
x * C Complex.({re = 2.; im = 3.});;
x - y;;
x / y;;
x *@ y;;

...

x > z;;
x >. z;;
(x >. z) * x;;
(x >. F 0.5) * x;;
(F 10. * x) + y *@ z;;

...

round (F 10. * (x *@ z));;
sin (F 5.) * cos (x + z);;
tanh (x * F 10. - z);;

...

Before we finish this chapter, I want to point out the caveat. Ext tries to mimic the dynamic languages like Python by with unified types. This prevents OCaml compiler from doing type checking in compilation phase and introduces extra overhead in calling functions. Therefore, besides fast experimenting in toplevel, I do not recommend to use Ext module in the production code.