Query Construction

Query Syntax and Execution

Two syntaxes are available for constructing queries: an “operator” syntax using Python’s comparators, and a “fluent” syntax where terms are chained together. Which to use is a matter of preference, and both construct the same query object.

Operator Syntax

Searches are built up from a series of Terminal nodes, which compare structural attributes to some search value. In the operator syntax, Python’s comparator operators are used to construct the comparison. The operators are overloaded to return Terminal objects for the comparisons.

Here is an example from the RCSB PDB Search API page created using the operator syntax. This query finds symmetric dimers having a twofold rotation with the DNA-binding domain of a heat-shock transcription factor.

from rcsbapi.search import TextQuery
from rcsbapi.search import search_attributes as attrs

# Create terminals for each query
q1 = TextQuery("heat-shock transcription factor")
q2 = attrs.rcsb_struct_symmetry.symbol == "C2"
q3 = attrs.rcsb_struct_symmetry.kind == "Global Symmetry"
q4 = attrs.rcsb_entry_info.polymer_entity_count_DNA >= 1

Attributes are available from the search_attributes object and can be tab-completed. They can additionally be constructed from strings using the Attr (attribute) constructor.

List of supported comparative operators:

Operator

Description

==

is

!=

is not

>

greater than

>=

greater than or equal to

<

less than

<=

less than or equal to

To use the "exists", "contains_phrase", or "contains_words" operator, create an AttributeQuery

For methods to search and find details on attributes within this package, go to the attributes page For a full list of attributes, please refer to the RCSB PDB schema.

Individual Terminals are combined into Groups using python’s bitwise operators. This is analogous to how bitwise operators act on python set objects. The operators are lazy and won’t perform the search until the query is executed.

query = q1 & (q2 & q3 & q4)  # AND of all queries

AND (&), OR (|), and terminal negation (~) are implemented directly by the API, but the python package also implements set difference (-), symmetric difference (^), and general negation by transforming the query.

List of supported bitwise operators:

Operator

Description

&

AND

|

OR

~

NOT

^

XOR/symmetric difference

-

set difference

Queries are executed by calling them as functions. They return an iterator of result identifiers.

# Call the query to execute it
results = query()

for rid in results:
    print(rid)

By default, the query will return “entry” results (PDB IDs corresponding to entries). It is also possible to query other types of results (see return-types for options):

# Set return_type to "assembly" when executing
results = query(return_type="assembly")

for assembly_id in results:
    print(assembly_id)

Fluent Syntax

The operator syntax is great for simple queries, but requires parentheses or temporary variables for complex nested queries. In these cases the fluent syntax may be clearer. Queries are built up by appending operations sequentially.

Here is the same example using the fluent syntax

from rcsbapi.search import TextQuery, AttributeQuery, Attr

# Start with a Attr or TextQuery, then add terms
results = TextQuery("heat-shock transcription factor").and_(
    # Add attribute node as fully-formed AttributeQuery
    AttributeQuery(
        attribute="rcsb_struct_symmetry.symbol",
        operator="exact_match",
        value="C2"
    )

    # Add attribute node as Attr object with chained operations
    # Setting type to "text" specifies that it's a Structure Attribute
    .and_(Attr(
        attribute="rcsb_struct_symmetry.kind",
        type="text"
    )).exact_match("Global Symmetry")

    # Add attribute node by name (converted to Attr object) with chained operations
    .and_("rcsb_entry_info.polymer_entity_count_DNA").greater_or_equal(1)

    # Execute the query and return assembly ids
    ).exec(return_type="assembly")

# Exec produces an iterator of IDs
for assembly_id in results:
    print(assembly_id)

Grouping Sub-Queries

Grouping of Structural Attribute and Chemical Attribute queries is permitted. More details on attributes that are available for attribute searches can be found on the RCSB PDB Search API page.

from rcsbapi.search import AttributeQuery

# Query for structures determined by electron microscopy
q1 = AttributeQuery(
    attribute="exptl.method",
    operator="exact_match",
    value="electron microscopy"
)

# Drugbank annotations contain phrase "tylenol"
q2 = AttributeQuery(
    attribute="drugbank_info.brand_names",
    operator="contains_phrase",
    value="tylenol"
)

# Combine queries with AND
query = q1 & q2

list(query())

Nested Attributes

Some attributes in the RCSB schema are part of a nested indexing context, meaning they must be queried as a pair–i.e., a group node containing only those two attributes–to ensure correct matching behavior. These include attributes like rcsb_binding_affinity.type and rcsb_binding_affinity.value, which are associated with the same underlying object (e.g., an EC50 measurement). As another example, the attribute rcsb_chem_comp_related.resource_name could be set to “DrugBank” or another database and grouped with the attribute rcsb_chem_comp_related.resource_accession_code, which can be used to search for an accession code. When grouped as a pair, these attributes will be searched for together (i.e. the accession code must be associated with the specified database).

To group such attributes correctly, use the NestedAttributeQuery class. This ensures the query is constructed in a way that complies with the schema’s nested indexing rules and returns the correct set of results.

More information about which attribute pairs support nested indexing can be found by inspecting the schema and looking at rcsb_nested_indexing_context fields of the attribute definitions.

from rcsbapi.search import AttributeQuery, NestedAttributeQuery

# Query for EC50-type binding affinity with a value of 2.0
q1 = AttributeQuery(
    attribute="rcsb_binding_affinity.type",
    operator="exact_match",
    value="EC50"
)

q2 = AttributeQuery(
    attribute="rcsb_binding_affinity.value",
    operator="equals",
    value=2.0
)

# Other independent structural filters
q3 = AttributeQuery(
    attribute="rcsb_entry_info.selected_polymer_entity_types",
    operator="exists"
)

# Group nested attributes using `NestedAttributeQuery`
query = NestedAttributeQuery(q1, q2) & q3

list(query())

If you do not use NestedAttributeQuery, your query may be “flattened”, leading to incorrect behavior or ignored nested constraints. Additionally, a warning message will appear, informing you of improper nested attribute usage.

For example, the effect of using NestedAttributeQuery can be visualized as follows:

# Without using `NestedAttributeQuery`
query = (q1 & q2) & q3

# Resulting JSON query submitted to search API:
query = {
    "operator": and,
    "nodes": [
        # Notice how all three nodes are in the same group/level
        q1,
        q2,
        q3
    ]
}


# WITH using `NestedAttributeQuery`
query = NestedAttributeQuery(q1, q2) & q3

# Resulting JSON query submitted to search API:
query = {
    "operator": and,
    "nodes": [
        # Now (q1 & q2) are paired together in their own dedicated group node
        {
            "operator": and,
            "nodes": [
                q1,
                q2
            ]
        },
        q3
    ]
}

# Importantly, this subtle difference can have a *BIG* impact on the query results!

Custom/forced sub-grouping attributes

The above NestedAttributeQuery class is intended for creating separated group nodes containing a single pair of nested attributes (and provides a validation of the attributes being paired to ensure that they are indeed nested attributes), to avoid flattening of otherwise equivalent AND and OR sub-queries (e.g., it prevents (q1 & q2) & q3 from becoming q1 & q2 & q3, keeping (q1 & q2) as a separate group from q3). However, if you are interested in forcing the sub-grouping of any other subset of attributes, you can do so using the group function.

from rcsbapi.search import AttributeQuery
from rcsbapi.search import group

q1 = AttributeQuery(
    attribute="exptl.method",
    operator="exact_match",
    value="electron microscopy"
)

q2 = AttributeQuery(
    attribute="rcsb_id",
    operator="exact_match",
    value="ATP",
    service="text_chem"
)

q3 = AttributeQuery(
    attribute="rcsb_entity_source_organism.scientific_name",
    operator="exact_match",
    value="Escherichia coli"
)

# Using `group` ensures that `exptl.method` and `rcsb_id` attributes are searched together
query = group(q1 & q2) & q3
print(list(query()))
print(query().get_editor_link())

Sessions

The result of executing a query (either by calling it as a function or using exec()) is a Session object. It implements __iter__, so it is usually treated as an iterator of IDs.

Paging is handled transparently by the session, with additional API requests made lazily as needed. The page size can be controlled with the rows parameter.

first = next(iter(query(rows=1)))

Progress Bar

The iquery() Session method provides a progress bar indicating the number of API requests being made.

results = query().iquery()

Search Service Types

The list of supported search service types are listed in the table below.

Search service

QueryType

Full-text

TextQuery()

Attribute (structure or chemical)

AttributeQuery()

Sequence similarity

SeqSimilarityQuery()

Sequence motif

SeqMotifQuery()

Structure similarity

StructSimilarityQuery()

Structure motif

StructMotifQuery()

Chemical similarity

ChemSimilarityQuery()

Learn more about available search services on the RCSB PDB Search API docs.

Full-Text Search

To perform a general search for structures associated with the phrase “Hemoglobin”, you can create a TextQuery. This does a “full-text” search, which is a general search on text associated with PDB structures or molecular definitions.

from rcsbapi.search import TextQuery

# Search for structures associated with the phrase "Hemoglobin"
query = TextQuery(value="Hemoglobin")

# Execute the query by running it as a function
results = query()

# Results are returned as an iterator of result identifiers.
for rid in results:
    print(rid)

Request Options

Return Types

A search query can return different results when a return_type is specified. Below are Structure Attribute query examples specifying return types "polymer_entity", "non_polymer_entity", "polymer_instance", and "mol_definition". More information on return types can be found in the RCSB PDB Search API page.

from rcsbapi.search import AttributeQuery

# query for 4HHB deoxyhemoglobin
q1 = AttributeQuery(
    attribute="rcsb_entry_container_identifiers.entry_id",
    operator="in",
    value=["4HHB"]
)

# Polymer entities
for poly in q1(return_type="polymer_entity"):
    print(poly)
    
# Non-polymer entities
for nonPoly in q1(return_type="non_polymer_entity"):
    print(nonPoly)
    
# Polymer instances
for polyInst in q1(return_type="polymer_instance"):
    print(polyInst)
    
# Molecular definitions
for mol in q1(return_type="mol_definition"):
    print(mol)

Computed Structure Models

The RCSB PDB Search API page provides information on how to include Computed Structure Models (CSMs) into a search query. Here is a code example below.

This query returns IDs for experimental and computational models associated with “hemoglobin”. Queries for only computed models or only experimental models can also be made (default).

from rcsbapi.search import TextQuery

q1 = TextQuery(value="hemoglobin")

# add parameter as a list with either "computational" or "experimental" or both
q2 = q1(return_content_type=["computational", "experimental"])

list(q2)

Results Verbosity

Results can be returned alongside additional metadata, including result scores. To return this metadata, set the results_verbosity parameter to "verbose" (all metadata), "minimal" (scores only), or "compact" (default, no metadata). If set to "verbose" or "minimal", results will be returned as a list of dictionaries.

For example, here we get all experimental models associated with “hemoglobin”, along with their scores, by setting verbosity to "minimal".

from rcsbapi.search import TextQuery

q1 = TextQuery(value="hemoglobin")
# Set results_verbosity to "minimal"
for idscore in list(q1(results_verbosity="minimal")):
    print(idscore)

Count Queries

If only the number of results is desired, the return_counts request option can be used. This query returns the number of experimental models associated with “hemoglobin”.

from rcsbapi.search import TextQuery

q1 = TextQuery(value="hemoglobin")

# Set return_counts request option to True
result_count = q1(return_counts=True)
print(result_count)

Faceted Queries

You can use a faceted query (or facets) to group and perform calculations and statistics on PDB data. Facets arrange search results into categories (buckets) based on the requested field values. More information on Faceted Queries can be found here. All facets should be provided with name, aggregation_type, and attribute values. Depending on the aggregation type, other parameters must also be specified. To run a faceted query, create a Facet object and pass it in as a single object or list into the facets argument during query execution.

from rcsbapi.search import AttributeQuery, Facet

q = AttributeQuery(
    attribute="rcsb_accession_info.initial_release_date",
    operator="greater",
    value="2019-08-20",
)

q_result = q(facets=Facet(
    name="Methods",
    aggregation_type="terms",
    attribute="exptl.method"
))
print(q_result.facets)

List of available types of Faceted queries:

  • Terms Facet

  • Histogram Facet

  • Range Facet

  • Date Range Facet

  • Cardinality Facet

  • Multidimensional Facet

  • Filter Facet

See example usage of each of these types of Faceted queries at Faceted Query Examples.

Additional Request Options

Other request options can also be added to queries through arguments at execution. facet, group_by, and sort are more complex request options and require creating a RequestOption object (Facet, GroupBy, Sort).

List of available request options:

  • results_content_type

  • results_verbosity

  • return_counts

  • facets

  • group_by

  • group_by_return_type

  • sort

  • return_explain_metadata

  • scoring_strategy

Some request options are currently not implemented:

  • paginate: Automatically handled by package. Results are paginated by package and all results are returned.

  • return_all_hits: Not implemented since all results are returned

For more information on what each request option does, refer to the Search API documentation.

For information on how to create RequestOption objects, see the API reference.