First Steps

Installing

Install the package with:

npm install transdb-de/filter-lang

You can now use it in your code with:

import * as filterLang from "@transdb-de/filter-lang"

Getting Started

First, you will need to define your language. The specification for a language definition can be found in the next section.

Then, you can construct a new language object, which will check your definition for errors, and convert it to an internal format, which can be used for parsing.

The language object can now parse strings, to the intermediate format, ready to be sent server side.

Using typescript, this can be achieved as follows:

import FilterLang from "@transdb-de/filter-lang";

const myDefinition: FilterLang.LanguageDefinition = [
  // your custom filter objects
];

const myLanguage = new FilterLang.Language(myDefinition);

const parsedObj = myLanguage.parse(userInput);

// ...

On the server, it can then be compiled, and sent to the Database:

import FilterLang from "@transdb-de/filter-lang";

const pipeline = FilterLang.Compiler.compileToMongoDB(parsedObj);

const results = await database.collection("myCollection").aggregate(pipeline).toArray();

Defining Your own Language

A filter-lang language definition is usually stored client side only, and defines how user input is translated to the intermediate format, in which it will then be sent to the server. These definitions are defined as an array of objects, each object describing a possible user input.

Simple Filter Example

Let's assume we have a database storing colors. We want to allow our users to filter these colors by name-search. In the database, the text name of a color is stored under info.color. Now we want our users to be able to filter the data in a format such as:

color: red

To parse this filter to an intermediate format which works with our database, we would need to following language definition:

const langDef = [
  {
    name: "Color",
    type: "text",
    field: "info.color"
  }
];

We defined one possible filter, allowing the user to filter for a colors name, by typing "color: name". type: "text" let's the filter-lang parser know, that we want this filter to be a text matcher with the values following the filter.

Wildcards

Everything that is not explicitly a filter, will be converted to a wildcard. Wildcards compile to a full text search, server-side. For example, lets say our color documents, have an additional field called: "manufacturer", which stores who produces this color. If we add a mongodb text index on this field, users can simply type a manufacturers name, and it will appear in the results.

Wildcards are useful in scenarios where a user might not be sure what field contains which values exactly. Applying text indexes on multiple fields, allows our end users to search by any string in those fields, without having to specify which property they are filtering.

Mappings

If our database contains unintuitive names, or we want our users to filter in a different language, we can proved mappings, to translate possible input values. In our example above, let's add a "info.type" field, which stores what this color is meant to be used for.

const langDef = [
  {
    name: "Type",
    type: "text",
    field: "info.type"
  }
];

Assuming we have colors for the types: "car", "wall", "roof", but in the database, we stored the values as "t_Car", "t_Wall", "t_Roof", we can now add the following mappings:

const langDef = [
  {
    name: "Type",
    type: "text",
    field: "info.type",
    mappings: {
      "car": "t_Car",
      "wall": "t_Wall",
      "roof": "t_Roof"
    }
  }
];

Our Users can now filter by usage type, using: type: car

Bulk Definitions

We can also use mappings to define a filter for multiple names and fields at once. Omitting the "name" and "field" fields, in a definition, will use the mappings keys as names, and it's values as fields.

We can also define a single filter, to apply to multiple fields at once, if we use "fields: [names]" instead of "field: name".

Suffixes

Some Filter types can be inverted by supplying a "negationSuffix".

Let's get back to our color example, and add the field: info.isLiquid to our database. This field can contain either "true", or "false". If we want our users to filter for liquid colors in the format: is: liquid, or is-not: liquid, we can write the following language definition:

const langDef = [
  {
    name: "is",
    negationSuffix: "not",
    type: "boolean",
    field: "info.isLiquid"
  }
];

Some Filters also accept multiple suffixes. These include date comparisons, and number comparisons.

Let's add the field: producedOn to our database, which stores a timestamp of the production date. We can let our users filter this filed by defining:

const langDef = [
  {
    name: "produced",
    type: "date-compare",
    suffixes: ["on", "before", "after"],
    field: "producedOn"
  }
];

Now our users can filter the database with filters like:

produced-on: 20.02.2018 all dates must be in "day month (year)" format

produced-before: 01.01.22 assumes current century was meant

produced-after: 31.03 assumes current year was meant

More Filter Types

Read on in the section Filter Types.