Search

Atlas exposes search over the metadata in two ways:

  • Search using DSL
  • Full-text search

Search DSL Grammar

The DSL exposes an SQL like query language for searching the metadata based on the type system. The grammar for the DSL is below.

queryWithPath: query ~ opt(WITHPATH)

query: rep1sep(singleQuery, opt(COMMA))

singleQuery: singleQrySrc ~ opt(loopExpression) ~ opt(selectClause)

singleQrySrc = FROM ~ fromSrc ~ opt(WHERE) ~ opt(expr ^? notIdExpression) |
        WHERE ~ (expr ^? notIdExpression) |
        expr ^? notIdExpression |
        fromSrc ~ opt(WHERE) ~ opt(expr ^? notIdExpression)

fromSrc: identifier ~ AS ~ alias | identifier

loopExpression: LOOP ~ (LPAREN ~> query <~ RPAREN) ~ opt(intConstant <~ TIMES) ~ opt(AS ~> alias)

selectClause: SELECT ~ rep1sep(selectExpression, COMMA)

selectExpression:  expr ~ opt(AS ~> alias)

expr:  compE ~ opt(rep(exprRight))

exprRight: (AND | OR) ~ compE

compE:
        arithE ~ (LT | LTE | EQ | NEQ | GT | GTE) ~ arithE |
            arithE ~ (ISA | IS) ~ ident  |
            arithE ~ HAS ~ ident  |
            arithE

arithE: multiE ~ opt(rep(arithERight))

arithERight: (PLUS | MINUS) ~ multiE

multiE: atomE ~ opt(rep(multiERight))

multiERight: (STAR | DIV) ~ atomE

atomE: literal | identifier | LPAREN ~> expr <~ RPAREN

identifier: rep1sep(ident, DOT)

alias: ident | stringLit

literal: booleanConstant |
        intConstant  |
        longConstant  |
        floatConstant |
        doubleConstant  |
        stringLit

Grammar language: {noformat} opt(a) => a is optional ~ => a combinator. 'a ~ b' means a followed by b rep => zero or more rep1sep => one or more, separated by second arg. {noformat}

Language Notes:

  • A SingleQuery expression can be used to search for entities of a Trait or Class.
Entities can be filtered based on a 'Where Clause' and Entity Attributes can be retrieved based on a 'Select Clause'.
  • An Entity Graph can be traversed/joined by combining one or more SingleQueries.
  • An attempt is made to make the expressions look SQL like by accepting keywords "SELECT",
"FROM", and "WHERE"; but these are optional and users can simply think in terms of Entity Graph Traversals.
  • The transitive closure of an Entity relationship can be expressed via the Loop expression. A
Loop expression can be any traversal (recursively a query) that represents a Path that ends in an Entity of the same Type as the starting Entity.
  • The WithPath clause can be used with transitive closure queries to retrieve the Path that
connects the two related Entities. (We also provide a higher level interface for Closure Queries see scaladoc for 'org.apache.metadata.query.ClosureQuery')
  • There are couple of Predicate functions different from SQL:
    • is or isacan be used to filter Entities that have a particular Trait.
    • has can be used to filter Entities that have a value for a particular Attribute.

DSL Examples

  • from DB
  • DB where name="Reporting" select name, owner
  • DB has name
  • DB is JdbcAccess
  • Column where Column isa PII
  • Table where name="sales_fact", columns
  • Table where name="sales_fact", columns as column select column.name, column.dataType, column.comment

Full-text Search

Atlas also exposes a lucene style full-text search capability.