- 
                Notifications
    
You must be signed in to change notification settings  - Fork 344
 
feat(modules): Add Cratedb container #888
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Open
      
      
            surister
  wants to merge
  8
  commits into
  testcontainers:main
  
    
      
        
          
  
    
      Choose a base branch
      
     
    
      
        
      
      
        
          
          
        
        
          
            
              
              
              
  
           
        
        
          
            
              
              
           
        
       
     
  
        
          
            
          
            
          
        
       
    
      
from
surister:feat/cratedb
  
      
      
   
  
    
  
  
  
 
  
      
    base: main
Could not load branches
            
              
  
    Branch not found: {{ refName }}
  
            
                
      Loading
              
            Could not load tags
            
            
              Nothing to show
            
              
  
            
                
      Loading
              
            Are you sure you want to change the base?
            Some commits from the old base branch may be removed from the timeline,
            and old review comments may become outdated.
          
          
      
        
          +360
        
        
          −3
        
        
          
        
      
    
  
  
     Open
                    Changes from all commits
      Commits
    
    
            Show all changes
          
          
            8 commits
          
        
        Select commit
          Hold shift + click to select a range
      
      4b58b9e
              
                Add first bare working implementation
              
              
                surister c92ad77
              
                Add as a module
              
              
                surister fbc47db
              
                Add tests
              
              
                surister 5d2616a
              
                Add more tests and `exposed_ports`
              
              
                surister 772a812
              
                Add entrypoint in example_basic.py
              
              
                surister 9410272
              
                Remove CRATEDB_DB as cratedb does not support different 'databases'
              
              
                surister 88938f0
              
                Remove dangling function
              
              
                surister 106fa5f
              
                Apply feedback from pr
              
              
                surister File filter
Filter by extension
Conversations
          Failed to load comments.   
        
        
          
      Loading
        
  Jump to
        
          Jump to file
        
      
      
          Failed to load files.   
        
        
          
      Loading
        
  Diff view
Diff view
There are no files selected for viewing
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| .. autoclass:: testcontainers.cratedb.CrateDBContainer | ||
| .. title:: testcontainers.cratedb.CrateDBContainer | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| import sqlalchemy | ||
| 
     | 
||
| from testcontainers import cratedb | ||
| 
     | 
||
| 
     | 
||
| def main(): | ||
| with cratedb.CrateDBContainer("crate:latest", ports={4200: None, 5432: None}) as container: | ||
| engine = sqlalchemy.create_engine(container.get_connection_url()) | ||
| with engine.begin() as conn: | ||
| result = conn.execute(sqlalchemy.text("select version()")) | ||
| version = result.fetchone() | ||
| print(version) | ||
| 
     | 
||
| 
     | 
||
| if __name__ == "__main__": | ||
| main() | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,155 @@ | ||
| import os | ||
| import typing as t | ||
| from urllib.parse import quote | ||
| 
     | 
||
| from testcontainers.core.container import DockerContainer | ||
| from testcontainers.core.exceptions import ContainerStartException | ||
| from testcontainers.core.utils import raise_for_deprecated_parameter | ||
| from testcontainers.core.wait_strategies import HttpWaitStrategy | ||
| 
     | 
||
| 
     | 
||
| class CrateDBContainer(DockerContainer): | ||
| """ | ||
| CrateDB database container. | ||
| 
     | 
||
| Example: | ||
| 
     | 
||
| The example spins up a CrateDB database and connects to it using | ||
| SQLAlchemy and its Python driver. | ||
| 
     | 
||
| .. doctest:: | ||
| 
     | 
||
| >>> from testcontainers import cratedb import CrateDBContainer | ||
| >>> import sqlalchemy | ||
| 
     | 
||
| >>> cratedb_container = | ||
| >>> with CrateDBContainer("crate:6.0") as cratedb: | ||
| ... engine = sqlalchemy.create_engine(cratedb.get_connection_url()) | ||
| ... with engine.begin() as connection: | ||
| ... result = connection.execute(sqlalchemy.text("select version()")) | ||
| ... version, = result.fetchone() | ||
| >>> version | ||
| 'CrateDB 6.0.2..' | ||
| """ | ||
| 
     | 
||
| CMD_OPTS: t.ClassVar[dict[str, str]] = { | ||
| "discovery.type": "single-node", | ||
| } | ||
| 
     | 
||
| def __init__( | ||
| self, | ||
| image: str = "crate/crate:latest", | ||
| ports: t.Optional[dict] = None, | ||
| user: t.Optional[str] = None, | ||
| password: t.Optional[str] = None, | ||
| cmd_opts: t.Optional[dict] = None, | ||
| **kwargs, | ||
| ) -> None: | ||
| """ | ||
| :param image: docker hub image path with optional tag | ||
| :param ports: optional dict that maps a port inside the container to a port on the host machine; | ||
| `None` as a map value generates a random port; | ||
| Dicts are ordered. By convention, the first key-val pair is designated to the HTTP interface. | ||
| Example: {4200: None, 5432: 15432} - port 4200 inside the container will be mapped | ||
| to a random port on the host, internal port 5432 for PSQL interface will be mapped | ||
| to the 15432 port on the host. | ||
| :param user: optional username to access the DB; if None, try `CRATEDB_USER` environment variable | ||
| :param password: optional password to access the DB; if None, try `CRATEDB_PASSWORD` environment variable | ||
| :param cmd_opts: an optional dict with CLI arguments to be passed to the DB entrypoint inside the container | ||
| :param kwargs: misc keyword arguments | ||
| """ | ||
| super().__init__(image=image, **kwargs) | ||
| cmd_opts = cmd_opts or {} | ||
| self._command = self._build_cmd({**self.CMD_OPTS, **cmd_opts}) | ||
| 
     | 
||
| self.CRATEDB_USER = user or os.environ.get("CRATEDB_USER", "crate") | ||
| self.CRATEDB_PASSWORD = password or os.environ.get("CRATEDB_PASSWORD", "crate") | ||
| 
     | 
||
| self.port_mapping = ports if ports else {4200: None} | ||
| self.port_to_expose = next(iter(self.port_mapping.items())) | ||
| 
     | 
||
| self.waiting_for(HttpWaitStrategy(4200).for_status_code(200).with_startup_timeout(5)) | ||
| 
     | 
||
| def exposed_ports(self) -> dict[int, int]: | ||
| """Returns a dictionary with the ports that are currently exposed in the container. | ||
| 
     | 
||
| Contrary to the '--port' parameter used in docker cli, this returns {internal_port: external_port} | ||
| 
     | 
||
| Examples: | ||
| {4200: 19382} | ||
| 
     | 
||
| :returns: The exposed ports. | ||
| """ | ||
| return {port: self.get_exposed_port(port) for port in self.ports} | ||
| 
     | 
||
| @staticmethod | ||
| def _build_cmd(opts: dict) -> str: | ||
| """ | ||
| Return a string with command options concatenated and optimised for ES5 use | ||
| """ | ||
| cmd = [] | ||
| for key, val in opts.items(): | ||
| if isinstance(val, bool): | ||
| val = str(val).lower() | ||
| cmd.append(f"-C{key}={val}") | ||
| return " ".join(cmd) | ||
| 
     | 
||
| def _configure_ports(self) -> None: | ||
| """ | ||
| Bind all the ports exposed inside the container to the same port on the host | ||
| """ | ||
| # If host_port is `None`, a random port to be generated | ||
| for container_port, host_port in self.port_mapping.items(): | ||
| self.with_bind_ports(container=container_port, host=host_port) | ||
| 
     | 
||
| def _configure_credentials(self) -> None: | ||
| self.with_env("CRATEDB_USER", self.CRATEDB_USER) | ||
| self.with_env("CRATEDB_PASSWORD", self.CRATEDB_PASSWORD) | ||
| 
     | 
||
| def _configure(self) -> None: | ||
| self._configure_ports() | ||
| self._configure_credentials() | ||
| 
     | 
||
| def get_connection_url(self, dialect: str = "crate", host: t.Optional[str] = None) -> str: | ||
| # We should remove this method once the new DBContainer generic gets added to the library. | ||
| """ | ||
| Return a connection URL to the DB | ||
| 
     | 
||
| :param host: optional string | ||
| :param dialect: a string with the dialect name to generate a DB URI | ||
| :return: string containing a connection URL to te DB | ||
| """ | ||
| return self._create_connection_url( | ||
| dialect=dialect, | ||
| username=self.CRATEDB_USER, | ||
| password=self.CRATEDB_PASSWORD, | ||
| host=host, | ||
| port=self.port_to_expose[0], | ||
| ) | ||
| 
     | 
||
| def _create_connection_url( | ||
| self, | ||
| dialect: str, | ||
| username: str, | ||
| password: str, | ||
| host: t.Optional[str] = None, | ||
| port: t.Optional[int] = None, | ||
| dbname: t.Optional[str] = None, | ||
| **kwargs: t.Any, | ||
| ) -> str: | ||
| if raise_for_deprecated_parameter(kwargs, "db_name", "dbname"): | ||
| raise ValueError(f"Unexpected arguments: {','.join(kwargs)}") | ||
| 
     | 
||
| if self._container is None: | ||
| raise ContainerStartException("container has not been started") | ||
| 
     | 
||
| host = host or self.get_container_host_ip() | ||
| assert port is not None | ||
| 
     | 
||
| port = self.get_exposed_port(port) | ||
| quoted_password = quote(password, safe=" +") | ||
| 
     | 
||
| url = f"{dialect}://{username}:{quoted_password}@{host}:{port}" | ||
| if dbname: | ||
| url = f"{url}/{dbname}" | ||
| return url | ||
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| import urllib.parse | ||
| import os | ||
| 
     | 
||
| import sqlalchemy | ||
| import pytest | ||
| 
     | 
||
| from testcontainers.cratedb import CrateDBContainer | ||
| 
     | 
||
| 
     | 
||
| @pytest.mark.parametrize("version", ["5.9", "5.10", "6.0", "latest"]) | ||
| def test_docker_run_cratedb_versions(version: str): | ||
| with CrateDBContainer(f"crate:{version}") as container: | ||
| engine = sqlalchemy.create_engine(container.get_connection_url()) | ||
| with engine.begin() as conn: | ||
| result = conn.execute(sqlalchemy.text("select 1+2+3+4+5")) | ||
| sum_result = result.fetchone()[0] | ||
| assert sum_result == 15 | ||
| 
     | 
||
| 
     | 
||
| @pytest.mark.parametrize( | ||
| "ports, expected", | ||
| [ | ||
| ({5432: None, 4200: None}, False), | ||
| ({5432: 5432, 4200: 4200}, {5432: 5432, 4200: 4200}), | ||
| ], | ||
| ) | ||
| def test_docker_run_cratedb_ports(ports, expected): | ||
| with CrateDBContainer("crate:latest", ports=ports) as container: | ||
| exposed_ports = container.exposed_ports() | ||
| assert len(exposed_ports) == 2 | ||
| assert all(map(lambda port: isinstance(port, int), exposed_ports)) | ||
| if expected: | ||
| assert exposed_ports == expected | ||
| 
     | 
||
| 
     | 
||
| def test_docker_run_cratedb_credentials(): | ||
| expected_user, expected_password, expected_port = "user1", "pass1", 4200 | ||
| expected_default_dialect, expected_default_host = "crate", "localhost" | ||
| expected_defined_dialect, expected_defined_host = "somedialect", "somehost" | ||
| os.environ["CRATEDB_USER"], os.environ["CRATEDB_PASSWORD"] = expected_user, expected_password | ||
| 
     | 
||
| with CrateDBContainer("crate:latest", ports={4200: expected_port}) as container: | ||
| url = urllib.parse.urlparse(container.get_connection_url()) | ||
| user, password = url.netloc.split("@")[0].split(":") | ||
| host, port = url.netloc.split("@")[1].split(":") | ||
| assert user == expected_user | ||
| assert password == expected_password | ||
| assert url.scheme == expected_default_dialect | ||
| assert host == expected_default_host | ||
| assert int(port) == expected_port | ||
| 
     | 
||
| url = urllib.parse.urlparse( | ||
| container.get_connection_url(dialect=expected_defined_dialect, host=expected_defined_host) | ||
| ) | ||
| host, _ = url.netloc.split("@")[1].split(":") | ||
| 
     | 
||
| assert url.scheme == expected_defined_dialect | ||
| assert host == expected_defined_host | ||
| 
     | 
||
| 
     | 
||
| @pytest.mark.parametrize( | ||
| "opts, expected", | ||
| [ | ||
| pytest.param( | ||
| {"indices.breaker.total.limit": "90%"}, | ||
| ("-Cdiscovery.type=single-node -Cindices.breaker.total.limit=90%"), | ||
| id="add_cmd_option", | ||
| ), | ||
| pytest.param( | ||
| {"discovery.type": "zen", "indices.breaker.total.limit": "90%"}, | ||
| ("-Cdiscovery.type=zen -Cindices.breaker.total.limit=90%"), | ||
| id="override_defaults", | ||
| ), | ||
| ], | ||
| ) | ||
| def test_build_command(opts, expected): | ||
| db = CrateDBContainer(cmd_opts=opts) | ||
| assert db._command == expected | 
      
      Oops, something went wrong.
        
    
  
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
Uh oh!
There was an error while loading. Please reload this page.