# Transforms

Though aggregation and projection is often much faster and less expensive that running a script transform, sometimes complex computation or data-interleaving is required that would be impractical to accomplish within a single script instance.

Transforms work by applying a script to an output cursor. Depending on the complexity and number of documents, a cursor transformation may be executed and re-queued multiple times before the cursor is finally exhausted.

When a running transform hits a timeout or ops threshold of 80%, it will stop iterating over the cursor, run the `after` finalizing method, store the `memo` object and re-queue for another run.

## class Transform

To create a transform, simply define a class annotated with `@transform`. All methods are optional.

```javascript
const { transform } = require('decorators-transform')

@transform
class Transformer {

  /**
   * Called if there is an error in the source operation. If the method
   * does not throw an error, Cortex throws the original error.
   *
   * @param err {Fault} thrown fault.
   */
  error(err) {
    throw err
  }

  /**
   * Called if the result is not a cursor (but a single result value).
   * The method must return a result if it's to be transformed. If the 
   * method does not return a value, Cortex outputs the original result value.
   *
   * @param result {*}
   * @returns {*}
   */
  result(result) {
    return result
  }

  /**
   * Cursor handler called only once and before all other processing.
   *
   * @param memo { Object } An empty object that can be modified and is saved 
   *        for the duration of the transformation.
   * @param cursor {Cursor} The output cursor.
   */
  beforeAll(memo, { cursor }) {
  }

  /**
   * Cursor handler called before each script run, and shares 
   * the current script context with each and after.
   *
   * @param memo { Object }
   * @param cursor {Cursor}
   */
  before(memo, { cursor }) {
  }

  /**
   * Cursor handler called with each cursor object. Returning undefined 
   * filters the object from the output stream.
   *
   * @param object { Object } the current output document
   * @param memo { Object }
   * @param cursor {Cursor}\
   * @returns {*} a value to be pushed to the output cursor.
   */
  each(object, memo, { cursor }) {
    return object
  }

  /**
   * Cursor handler called after before() and each() was called.
   *
   * @param memo { Object }
   * @param cursor {Cursor}
   * @param count { Number } The number of documents pushed by the script 
   *        runtime during the current execution.
   */
  after(memo, { cursor, count }) {
  }

  /**
   * Cursor handler called only once and after the cursor is exhausted.
   *
   * @param memo { Object }
   * @param cursor {Cursor}
   */
  afterAll(memo, { cursor }) {
  }

}
```

## Types

### Policy Transform

A policy transform can handle a result value as well as a cursor. The define one, set the policy `action` to `"Transform"` and the `script` value to a transform export.

`script.arguments`:

* policy { Object }
  * \_id { ObjectID } The policy id
  * name { String } The policy name
* policyOptions { Object }
  * triggered {String\[]} The list of triggered conditions (methods, paths, etc.)�

```javascript
const { transform } = require('decorators-transform')

@transform
class PolicyTransform {

  error(err) {
    throw Fault.create('app.error.policyError', { faults: [err] })
  }

  result(result) {
    result.policyTriggers = script.arguments.policyOptions.triggered
    return result
  }

}

module.exports = PolicyTransform
```

### View Transform

A view transform acts on the result form a view cursor. The define one, set the `script` value to a transform export.

`script.arguments`:

* view { Object }
  * \_id { ObjectID } The view id
  * name { String } The view name
* viewOptions { Object } The original view cursor options (object, skipAcl, paths, pipeline, etc.)

```javascript
const { transform } = require('decorators-transform')

@transform
class ViewTransform {
  beforeAll(memo, { cursor }) {

    const { arguments: { view, viewOptions } } = script,
          { paths, object } = viewOptions

    cursor.push({
      object: 'transform',
      view, 
      viewOptions      
    })    
  }
}

module.exports = ViewTransform
```

### Inline Cursor Transform

A transform may be applied to any QueryCursor, AggregationCursor, or top-level BulkOperation.

`script.arguments`:

* cursorOptions { Object } The original cursor options (object, skipAcl, paths, pipeline, etc.)

There are a few ways to define a transform for a cursor.

A series of implicitly usable transform methods:

```javascript
const { c_datum: Datum } = org.objects

return Datum.find().transform(`
  beforeAll(memo, { cursor }) {
    Object.assign(memo, {
      object: 'resultTotals',
      sum: 0,
      deviceTypes: [],
      culled: 0
    })
  }
  each(object, memo) {
    if (object.c_inactive) {
      memo.culled += 1
      return // returning undefined filters the object from the output stream
    }
    memo.sum += object.c_amount || 0
    if (!memo.deviceTypes.includes(object.c_deviceType)) {
      memo.deviceTypes.push(object.c_deviceType)
    }
    return object // return the object to include it in the output stream.
  }
  afterAll(memo, { cursor }) {
    cursor.push(memo)
  }
`)
```

An implicit transform class:

```javascript
const { c_datum: Datum } = org.objects

return Datum.find().transform(`
  class {
    constructor() {
        this.runtTimestamp = Date.now()
    }
    each(object) {
        object.runtTimestamp = this.runtTimestamp
        return object
    }
  }
`)
```

A reference to a Library Script that exports an annotated transform class:

```javascript
const { c_datum: Datum } = org.objects

return Datum.find().transform('c_datum_transformer')
```

An inline export definition:

```javascript
const { c_datum: Datum } = org.objects

return Datum.find().transform({
  script: `
    const { transform } = require('decorators-transform'),
          { Accounts } = org.objects
    @transform
    class Transformer {
      each(object, memo, { cursor, index }) {
        return object._id
      }
    }
    module.exports = Transformer
  `
})
```

## Bulk Operations

Transformations can be applied to top-level bulk operations only.

```javascript
const { accounts: Datum, accounts: Other, accounts: Device } = org.objects,
      _id = Device
        .readOne({owner: script.principal._id})
        .execute()._id // 1 device per account.

return org.objects.bulk()
  .add(
    Datum.find({ c_device: _id }).limit(10)
  )
  .add(
    Other.find({ c_device: _id }).limit(10).transform('c_some_transform')
  )
  .transform(`
    each(object) {
      return object.data // unwrap
    }
  `)
```

## Examples

### CSV View Transform

```javascript
const { transform } = require('decorators-transform'),
      schemas = require('schemas'),
      res = require('response')

let Undefined

// @transform
class CSVTransform {

  constructor() {
    this.quote = '"'
    this.doubleQuote = '""'
  }

  beforeAll(memo) {

    const { arguments: { view, viewOptions } } = script,
          { paths, object } = viewOptions

    memo.fields = paths || schemas.read(object, 'properties').filter(v => !v.optional).map(v => v.name)

    res.setHeader('Content-Type', 'text/csv')
    res.write('#view: ' + JSON.stringify(view) + '\n')
    res.write('#viewOptions: ' + JSON.stringify(viewOptions) + '\n')
    res.write(memo.fields.join(','))
  }

  each(object, memo) {
    res.write(
      '\n' +
      memo.fields.reduce((row, field) => {
        row.push(this.toCSV(object[field]))
        return row
      }, []).join(',')
    )
  }

  toCSV(value) {
    if (value === null || value === Undefined) {
      return Undefined
    }
    const valueType = typeof value
    if (valueType !== 'boolean' && valueType !== 'number' && valueType !== 'string') {
      value = JSON.stringify(value)
      if (value === Undefined) {
        return Undefined
      }
      if (value[0] === this.quote) {
        value = value.replace(/^"(.+)"$/, '$1')
      }
    }
    if (typeof value === 'string') {
      if (value.includes(this.quote)) {
        value = value.replace(new RegExp(this.quote, 'g'), this.doubleQuote)
      }
      value = this.quote + value + this.quote
    }
    return value
  }

}

module.exports = CSVTransform
```

### Inline CSV Transform

```javascript
const { c_datum: Datum } = org.objects

return Datum.find().transform(`
  quote = '"'
  doubleQuote = '""'
  res = require('response)
  beforeAll(memo) {
    const schemas = require('schemas'),
          { arguments: { cursorOptions: { Object } } } = script
    memo.fields = schemas.read(object, 'properties').filter(v => !v.optional).map(v => v.name)
    this.res.setHeader('Content-Type', 'text/csv')
    this.res.write(memo.fields.join(','))
  }
  each(object, memo) {
    this.res.write(
      '\\n' +
      memo.fields.reduce((row, field) => {
        row.push(this.toCSV(object[field]))
        return row
      }, []).join(',')
    )
  }
  toCSV(value) {
    if (value === null || value === undefined) {
      return undefined
    }
    const valueType = typeof value
    if (valueType !== 'boolean' && valueType !== 'number' && valueType !== 'string') {
      value = JSON.stringify(value)
      if (value === undefined) {
        return undefined
      }
      if (value[0] === this.quote) {
        value = value.replace(/^"(.+)"$/, '$1')
      }
    }
    if (typeof value === 'string') {
      if (value.includes(this.quote)) {
        value = value.replace(new RegExp(this.quote, 'g'), this.doubleQuote)
      }
      value = this.quote + value + this.quote
    }
    return value
  }
`)
```

### Account Policy

This example triggers the policy on any individual account read

```javascript
// Triggers on GET /account(s)?/([a-fa-f0-9]{24})

const { transform } = require('decorators-transform'),
      { equalIds } = require('util.id')

let Undefined

@transform
class PolicyTransform {

  // throw generic opaque error
  error(err) {
    throw Fault.create('app.error.accountAccess')
  }

  // throw access error even if the caller has access to the account
  // this might be the result of privileged JWT.
  result(result) {
    if (!equalIds(script.principal._id, result._id)) {
      throw Fault.create('app.accessDenied.account')
    }
    // add the policy triggers to the result for fun and profit
    // in this case: ['methods', 'paths']
    result.policyTriggers = script.arguments.policyOptions.triggered
    return result
  }

}

module.exports = PolicyTransform
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.medable.com/cortex-api/scripting/transforms.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.