profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/jsravn/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

jsravn/dotfiles 3

my nixos configuration

jsravn/emacs-config 1

my emacs configuration

jsravn/fixpixel 1

Terminal app that spams RGB signals to fix stuck pixels.

jsravn/advent2017 0

solutions for https://adventofcode.com/2017

jsravn/akka 0

Build highly concurrent, distributed, and resilient message-driven applications on the JVM

jsravn/akka-kubernetes-tests 0

Basic tests that show Akka Cluster and friends work on Kubernetes

jsravn/ansible-modules-core 0

Ansible modules - these modules ship with ansible

jsravn/assertj-core 0

AssertJ is a library of assertions similar to fest-assert but providing a richer set of assertions.

jsravn/autoscaler 0

Customised fork of cluster-autoscaler to support machine-controller-manager

fork Skarlso/apis

Public API definitions of ArangoDB Managed Services

https://cloud.arangodb.com

fork in 2 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void shouldNotLoadAnyOfTheCqlFilesIfThereAreDuplicateCqlFilenames() throw          //when         try {-            MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+            MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);             fail("Should have died");         } catch (RuntimeException e) {             // nada         }          //then-        KeyspaceMetadata keyspaceMetadata = cluster.getMetadata().getKeyspace(TEST_KEYSPACE);-        assertThat(keyspaceMetadata).as("should not have made any schema changes").isNull();+        Optional<KeyspaceMetadata> keyspaceMetadata = session.getMetadata().getKeyspace(TEST_KEYSPACE);+        assertThatThrownBy(keyspaceMetadata::get).as("should not have made any schema changes").isInstanceOf(NoSuchElementException.class);

Can just assert that keyspaceMetadata is empty?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public boolean release(String clientId) throws CannotReleaseLockException {             ResultSet resultSet = session.execute(deleteLockQuery.bind(lockName, clientId));             Row result = resultSet.one(); -            // if a row doesn't exist then cassandra doesn't send back any columns-            boolean noLockExists = !result.getColumnDefinitions().contains("client");-            if (result.getBool("[applied]") || noLockExists) {-                log.info("Lock released for {} by client {} at: {}", lockName, clientId, System.currentTimeMillis());-                return true;-            }--            String clientReleasingLock = result.getString("client");-            if (!clientReleasingLock.equals(clientId)) {-                if (isRetryAfterWriteTimeout) {-                    log.info("Released lock for client {} in retry attempt after WriteTimeoutException", clientReleasingLock);+            if (result != null) {+                // if a row doesn't exist then cassandra doesn't send back any columns

Remember how this was found/proven? The only test we have is mocked and curious how this was discovered

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public boolean acquire(String clientId) throws CannotAcquireLockException {             ResultSet resultSet = session.execute(insertLockQuery.bind(lockName, clientId));             Row currentLock = resultSet.one();             // we could already hold the lock and not be aware if a previous acquire had a writetimeout as a timeout is not a failure in cassandra-            if (currentLock.getBool("[applied]") || clientId.equals(currentLock.getString("client"))) {+            //TODO this needs to be tested+            if (currentLock != null && (currentLock.getBoolean("[applied]") || clientId.equals(currentLock.getString("client")))) {

Still needs testing?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void rowInsertedWithMessageDigestHashingAlgorithmIsSameAsGuavaSha1Hashing          //when         schemaUpdates.add(-            filename,-            Paths.get(cqlResource.toURI())+                filename,+                Paths.get(cqlResource.toURI())         );          //then         final String guavaSha1Hash = Resources.asByteSource(cqlResource).hash(Hashing.sha1()).toString();         final ResultSet resultSet = session.execute("SELECT * from " + SCHEMA_UPDATES_TABLE);-        assertThat(resultSet.all())-            .hasOnlyOneElementSatisfying(row -> {-                assertThat(row.getString("filename"))+        assertThat(resultSet.all().size()).isEqualTo(1);+        resultSet.forEach(row ->+        {+            assertThat(row.getString("filename"))                     .isEqualTo(filename);-                assertThat(row.getString("checksum"))+            assertThat(row.getString("checksum"))                     .isEqualTo(guavaSha1Hash);-            });+        });     }

any reason to change this out of curiosity? Don't mind either way.

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void tearDown() {     }      @AfterClass-    public static void tearDownClass() throws Exception {+    public static void tearDownClass() {         EmbeddedCassandraServerHelper.cleanEmbeddedCassandra();         Uninterruptibles.sleepUninterruptibly(100, TimeUnit.MILLISECONDS);     }      @Test-    public void schemaUpdatesTableShouldBeCreatedIfNotExists() throws Exception {+    public void schemaUpdatesTableShouldBeCreatedIfNotExists() {         //given-        cluster.connect("system").execute("CREATE KEYSPACE " + TEST_KEYSPACE + " WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1 };");-        Session session = cluster.connect(TEST_KEYSPACE);         SessionContext sessionContext = new SessionContext(session, ConsistencyLevel.ALL, ConsistencyLevel.ALL, clusterHealth);         SchemaUpdates schemaUpdates = new SchemaUpdates(sessionContext, TEST_KEYSPACE);          //when         schemaUpdates.initialise();          //then-        KeyspaceMetadata keyspaceMetadata = cluster.getMetadata().getKeyspace(TEST_KEYSPACE);+        KeyspaceMetadata keyspaceMetadata = session.getMetadata().getKeyspace(TEST_KEYSPACE).get();         assertThat(keyspaceMetadata.getTable(SCHEMA_UPDATES_TABLE)).as("table should have been created").isNotNull();

can check optional is empty? if the test fails the .get() with throw an exception rather than fail this assertion.

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void shouldNotRemoveLockAfterMigrationFailed() throws Exception {          //when         try {-            MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+            MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);         } catch (RuntimeException ignore) {         }          //then-        ResultSet resultSet = session.execute("SELECT * FROM cqlmigrate.locks WHERE name = ?", LOCK_NAME);-        assertThat(resultSet.isExhausted()).as("Is lock released").isFalse();+        ResultSet resultSet = session.execute(newInstance("SELECT * FROM cqlmigrate.locks WHERE name = ?", LOCK_NAME));+        assertThat(resultSet.one()).as("Is lock released").isNotNull();     }      @Test     public void shouldRemoveLockAfterMigrationFailedIfUnlockOnFailureIsSetToTrue() throws Exception {         //given         CqlMigrator migrator = new CqlMigratorImpl(CqlMigratorConfig.builder()-            .withLockConfig(CassandraLockConfig.builder().unlockOnFailure().build())-            .withReadConsistencyLevel(ConsistencyLevel.ALL)-            .withWriteConsistencyLevel(ConsistencyLevel.ALL)-            .build(), new SessionContextFactory());+                .withLockConfig(CassandraLockConfig.builder().unlockOnFailure().build())+                .withReadConsistencyLevel(ConsistencyLevel.ALL)+                .withWriteConsistencyLevel(ConsistencyLevel.ALL)+                .build(), new SessionContextFactory());         Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap_missing_semicolon"));          //when         try {-            migrator.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+            migrator.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);         } catch (RuntimeException ignore) {         }          //then-        ResultSet resultSet = session.execute("SELECT * FROM cqlmigrate.locks WHERE name = ?", LOCK_NAME);-        assertThat(resultSet.isExhausted()).as("Is lock released").isTrue();+        ResultSet resultSet = session.execute(newInstance("SELECT * FROM cqlmigrate.locks WHERE name = ?", LOCK_NAME));+        assertThat(resultSet.one()).as("Is lock released").isNull();     }      @Test     public void shouldRetryWhenAcquiringLockIfNotInitiallyAvailable() throws Exception {         //given-        session.execute("INSERT INTO cqlmigrate.locks (name, client) VALUES (?, ?)", LOCK_NAME, UUID.randomUUID().toString());+        session.execute(newInstance("INSERT INTO cqlmigrate.locks (name, client) VALUES (?, ?)", LOCK_NAME, UUID.randomUUID().toString()));         Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap"));          //when         Future<?> future = executorService.submit(() -> MIGRATOR.migrate(-                CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths));+                CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths));         session.execute("TRUNCATE cqlmigrate.locks");         Thread.sleep(1000);         future.get();          //then         try {-            cluster.connect(TEST_KEYSPACE);+            session.getMetadata().getKeyspace(TEST_KEYSPACE);

Are these checking the same thing? getKeyspace() returns an optional which could still be empty if the bootstrap hasn't been applied?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

     private static final String KEYSPACE = "locker";     private static String TABLE_NAME = KEYSPACE + ".lock_testing"; -    private Cluster cluster;-    private Session session;+    private static Server server = Server.builder().build();+    private ClusterSpec cluster = ClusterSpec.builder().build();+    private BoundCluster bCluster;

Now that this test uses simulacron rather than real cassandra, could the @ignore be removed? Can it run in pipelines now?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void shouldObtainLockToInsertRecord() throws InterruptedException {     }      private void updateCounter(int updatedCounter) {-        Statement updateCounterQuery = new SimpleStatement(String.format("UPDATE %s SET counter = %d WHERE id = 'lock-tester'", TABLE_NAME, updatedCounter)).setConsistencyLevel(ConsistencyLevel.QUORUM);+        Statement updateCounterQuery = SimpleStatement.newInstance(String.format("UPDATE %s SET counter = %d WHERE id = 'lock-tester'", TABLE_NAME, updatedCounter)).setConsistencyLevel(ConsistencyLevel.QUORUM);         session.execute(updateCounterQuery);      }      private int readCurrentCounter() {-        Statement readCounterQuery = new SimpleStatement(String.format("SELECT counter from %s WHERE id = 'lock-tester'", TABLE_NAME)).setConsistencyLevel(ConsistencyLevel.QUORUM);+        Statement readCounterQuery = SimpleStatement.newInstance(String.format("SELECT counter from %s WHERE id = 'lock-tester'", TABLE_NAME)).setConsistencyLevel(ConsistencyLevel.QUORUM);         final ResultSet currentRecord = session.execute(readCounterQuery);         return currentRecord.one().getInt("counter");     } -    private Cluster createCluster() {-        QueryOptions queryOptions = new QueryOptions();-        queryOptions.setConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM);--        SocketOptions socketOptions = new SocketOptions();-        socketOptions.setReadTimeoutMillis(1000);--        return Cluster.builder()-                .addContactPoints(CASSANDRA_HOST)-                .withPort(CASSANDRA_PORT)-                .withQueryOptions(queryOptions)-                .withSocketOptions(socketOptions)-                .build();-    }

We no longer specify timeouts in this test. Were they needed?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void shouldThrowExceptionForInvalidBootstrap() throws Exception {         Collection<Path> cqlPaths = singletonList(cqlPath);          //when-        MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+        MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);          //then-        cluster.connect(TEST_KEYSPACE);+        //throws exception

Could we catch the throwable and assert it's type instead of nothing? This has highlighted that it's not obvious which functional call the exception was previously being thrown from

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void schemaUpdatesTableShouldNotBeCreatedIfExists() throws Exception {             fail("Expected " + SCHEMA_UPDATES_TABLE + " table creation to be attempted only once.");         }         //then-        KeyspaceMetadata keyspaceMetadata = cluster.getMetadata().getKeyspace(TEST_KEYSPACE);+        KeyspaceMetadata keyspaceMetadata = session.getMetadata().getKeyspace(TEST_KEYSPACE).get();         assertThat(keyspaceMetadata.getTable(SCHEMA_UPDATES_TABLE)).as("table should have been created").isNotNull();

can check optional is empty? if the test fails the .get() with throw an exception rather than fail this assertion.

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void migrate(Session session, String keyspace, Collection<Path> directori     /**      * {@inheritDoc}      */-    public void clean(String[] hosts, int port, String username, String password, String keyspace) {-        try (Cluster cluster = CassandraClusterFactory.createCluster(hosts, port, username, password);-             Session session = cluster.connect()) {-            this.clean(session, keyspace);+    public void clean(String[] hosts, String localDC, int port, String username, String password, String keyspace) {++        List<InetSocketAddress> cassandraHosts = Stream.of(hosts).map(host -> new InetSocketAddress(host, port)).collect(Collectors.toList());++        try (CqlSession cqlSession = CqlSession.builder()

(mentioned earlier) no longer using the factory to create the session; deliberate?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void shouldThrowExceptionWhenMultipleBootstrap() throws Exception {         Collection<Path> cqlPaths = asList(getResourcePath("cql_bootstrap"), getResourcePath("cql_bootstrap_duplicate"));          //when-        MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+        MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);          //then-        cluster.connect(TEST_KEYSPACE);+        //throws exception

Could we catch the throwable and assert it's type instead of nothing? This has highlighted that it's not obvious which functional call the exception was previously being thrown from

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public static void main(String[] args) {                 .collect(Collectors.toList());          CqlMigratorFactory.create(CassandraLockConfig.builder().build())-                .migrate(hosts.split(","), port == null ? 9042 : Integer.parseInt(port), username, password, keyspace, directories);+                .migrate(hosts.split(","), localDC, port == null ? 9042 : Integer.parseInt(port), username, password, keyspace, directories);     }      /**      * {@inheritDoc}      */-    public void migrate(String[] hosts, int port, String username, String password, String keyspace, Collection<Path> directories) {-        try (Cluster cluster = CassandraClusterFactory.createCluster(hosts, port, username, password);-             Session session = cluster.connect()) {-            this.migrate(session, keyspace, directories);+    public void migrate(String[] hosts, String localDC, int port, String username, String password, String keyspace, Collection<Path> directories) {+        List<InetSocketAddress> cassandraHosts = Stream.of(hosts).map(host -> new InetSocketAddress(host, port)).collect(Collectors.toList());++        try (CqlSession cqlSession = CqlSession.builder()

(mentioned earlier) no longer using the factory to create the session; deliberate?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public class ClusterHealthTest {      private static final int defaultStartingPort = PortScavenger.getFreePort();+    private static final Server server = Server.builder().build(); -    private ClusterHealth clusterHealth;+    private final ClusterSpec cluster = ClusterSpec.builder().build(); -    private static Server server = Server.builder().build();-    private ClusterSpec cluster = ClusterSpec.builder().build();+    private ClusterHealth clusterHealth;     private BoundCluster bCluster;--    private Cluster realCluster;+    private CqlSession realCluster;      @Before     public void setUp() throws UnknownHostException {         DataCenterSpec dc = cluster.addDataCenter().withName("DC1").withCassandraVersion("3.11").build();-        dc.addNode().withAddress( new InetSocketAddress(Inet4Address.getByAddress(new byte[] {127, 0, 0, 1}), defaultStartingPort)).build();+        dc.addNode().withAddress(new InetSocketAddress(Inet4Address.getByAddress(new byte[]{127, 0, 0, 1}), defaultStartingPort)).build();         dc.addNode().withPeerInfo("host_id", UUID.randomUUID()).build();         bCluster = server.register(cluster);          bCluster.prime(when("select cluster_name from system.local where key = 'local'")                 .then(rows().row("cluster_name", "0").build())); -        realCluster = CassandraClusterFactory.createCluster(new String[]{"localhost"}, defaultStartingPort, null, null);-        realCluster.connect();+        realCluster = CqlSession.builder()+                .addContactPoint(new InetSocketAddress(Inet4Address.getByAddress(new byte[]{127, 0, 0, 1}), defaultStartingPort))+                .withLocalDatacenter(dc.getName())+                .build();

Same question about not using the factory for creating cluster/session?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void shouldDeleteLockWhenReleasingLock() throws Exception {         //then         assertThat(isLockReleased).isTrue();         List<QueryLog> queryLogs = cluster.getLogs().getQueryLogs().stream()-            .filter(queryLog -> queryLog.getFrame().message.toString().contains(PREPARE_DELETE_QUERY))-            .collect(Collectors.toList());+                .filter(queryLog -> queryLog.getFrame().message.toString().contains(PREPARE_DELETE_QUERY))+                .collect(Collectors.toList());          assertThat(queryLogs.size()).isEqualTo(1);      }      @Test-    public void shouldSuccessfullyReleaseLockWhenNoLockFound() throws Exception {+    public void shouldSuccessfullyReleaseLockWhenNoLockFound() {         cluster.prime(primeDeleteQuery(LOCK_KEYSPACE, CLIENT_ID, false));         lockingMechanism.init();         //when

What is different between this test and shouldReturnTrueIfNoLockExists from a functional perspective?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

     }      void initialise() {-        Session session = sessionContext.getSession();-        session.execute(new SimpleStatement("USE " + keyspace + ";").setConsistencyLevel(sessionContext.getReadConsistencyLevel()));-        TableMetadata schemaUpdateTableMetadata = session.getCluster().getMetadata().getKeyspace(keyspace).getTable(SCHEMA_UPDATES_TABLE);+        CqlSession session = sessionContext.getSession();+        session.execute(SimpleStatement.newInstance("USE " + keyspace + ";").setConsistencyLevel(sessionContext.getReadConsistencyLevel()));+        TableMetadata schemaUpdateTableMetadata = session.getMetadata().getKeyspace(keyspace).get().getTable(SCHEMA_UPDATES_TABLE).orElse(null);

Since we have optionals could we use them? getKeyspace(XXX).map(k -> k.getTable()).orElse(null)? While functionally the keyspace should exist, the .get() call is unchecked and could result in future issues.

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

      void bootstrap() {         Session session = sessionContext.getSession();-        KeyspaceMetadata keyspaceMetadata = session.getCluster().getMetadata().getKeyspace(keyspace);+        KeyspaceMetadata keyspaceMetadata = session.getMetadata().getKeyspace(keyspace).orElse(null);         if (keyspaceMetadata == null) {

Could we make use of optional isPresent rather than mapping to null and immediately checking for null?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void shouldMigrateSchemaIfLockCanBeAcquired() throws Exception {         Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap"));          //when-        MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+        MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);          //then         try {-            cluster.connect(TEST_KEYSPACE);+            session.getMetadata().getKeyspace(TEST_KEYSPACE);

Are these checking the same thing? getKeyspace() returns an optional which could still be empty if the bootstrap hasn't been applied?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 CassandraLockConfig lockConfig = CassandraLockConfig.builder()         .withLockKeyspace("cqlmigrate")         .build(); -// Create a Cassandra session-Cluster cluster = Cluster.builder.addContactPoint("localhost").build();-Session session = cluster.connect();+// Create a Cassandra session for cassandra driver 4.x+CqlSession session = CqlSession.builder()+        .addContactPoints(cassandraHosts)+        .withLocalDatacenter("datacenter1")+        .withAuthProvider(new ProgrammaticPlainTextAuthProvider("username", "password"))+        .build();

nitpick, but the import in this code snippet is no longer accurate

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void shouldRunTheBootstrapCqlIfKeyspaceDoesNotExist() throws Exception {         Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap"));          //when-        MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+        MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);          //then         try {-            cluster.connect(TEST_KEYSPACE);+            session.getMetadata().getKeyspace(TEST_KEYSPACE);

Are these checking the same thing? getKeyspace() returns an optional which could still be empty if the bootstrap hasn't been applied?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 package uk.sky.cqlmigrate; -import com.datastax.driver.core.Cluster;-import com.datastax.driver.core.Host;+import com.datastax.oss.driver.api.core.metadata.Node;+import com.datastax.oss.driver.api.core.metadata.NodeState;+import com.datastax.oss.driver.api.core.session.Session; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import uk.sky.cqlmigrate.exception.ClusterUnhealthyException;  import java.net.InetAddress; import java.util.List;-import java.util.Set;+import java.util.Map;+import java.util.UUID; import java.util.stream.Collectors;  class ClusterHealth {      private static final Logger log = LoggerFactory.getLogger(ClusterHealth.class); -    private final Cluster cluster;+    private final Session session; -    ClusterHealth(Cluster cluster) {-        this.cluster = cluster;+    ClusterHealth(Session session) {+        this.session = session;     }      void check() throws ClusterUnhealthyException { -        log.debug("Checking cluster health");+        log.debug("Checking session health"); -        Set<Host> allHosts = cluster.getMetadata().getAllHosts();+        Map<UUID, Node> nodes = session.getMetadata().getNodes(); -        List<InetAddress> unhealthyHosts = allHosts-                .stream()-                .filter(host -> !host.isUp())-                .map(Host::getAddress)+        List<InetAddress> unhealthyHosts = nodes.values().stream()+                .filter(node -> node.getState().equals(NodeState.DOWN))+                .map(node -> node.getBroadcastAddress().get().getAddress())                 .collect(Collectors.toList());          if (!unhealthyHosts.isEmpty()) {             throw new ClusterUnhealthyException("Cluster not healthy, the following hosts are down: " + unhealthyHosts);         } -        log.debug("All hosts healthy: {}", allHosts);+        log.debug("All hosts healthy: {}", nodes.entrySet());

This will log a bunch of UUIDs which may not be that useful; should it log the values() instead?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

      * and manual intervention is required to cleanup and fix the issue.      *      * @param hosts       Comma separated list of cassandra hosts+     * @param localDC     Local Datacenter name to be provided to create a session for cassandra driver 4.x.x      * @param port        Native transport port for the above cassandra nodes      * @param username    Username for Cassandra's internal authentication using PasswordAuthenticator      *                    (if using AllowAllAuthenticator, can be set to any value)      * @param password    Password for Cassandra's internal authentication using PasswordAuthenticator      *                    (if using AllowAllAuthenticator, can be set to any value)      * @param keyspace    Keyspace name for which the schema migration needs to be applied      * @param directories Comma separated list of directory paths containing the cql statements for the schema change-     * @throws ClusterUnhealthyException                           if any nodes are down or the schema is not in agreement before running migration-     * @throws CannotAcquireLockException                          if any of the queries to acquire lock fail or-     *                                                             {@link CassandraLockConfig.CassandraLockConfigBuilder#withTimeout(Duration)}-     *                                                             is reached before lock can be acquired.-     * @throws CannotReleaseLockException                          if any of the queries to release lock fail-     * @throws IllegalArgumentException                            if any file types other than .cql are found-     * @throws IllegalStateException                               if cql file has changed after migration has been run-     * @throws com.datastax.driver.core.exceptions.DriverException if any of the migration queries fails+     * @throws ClusterUnhealthyException                        if any nodes are down or the schema is not in agreement before running migration+     * @throws CannotAcquireLockException                       if any of the queries to acquire lock fail or+     *                                                          {@link CassandraLockConfig.CassandraLockConfigBuilder#withTimeout(Duration)}+     *                                                          is reached before lock can be acquired.+     * @throws CannotReleaseLockException                       if any of the queries to release lock fail+     * @throws IllegalArgumentException                         if any file types other than .cql are found+     * @throws IllegalStateException                            if cql file has changed after migration has been run+     * @throws com.datastax.oss.driver.api.core.DriverException if any of the migration queries fails

Since we're updating javadoc (albein whitespace) could we remove the fully qualified name and import instead?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public boolean release(String clientId) throws CannotReleaseLockException {             ResultSet resultSet = session.execute(deleteLockQuery.bind(lockName, clientId));             Row result = resultSet.one(); -            // if a row doesn't exist then cassandra doesn't send back any columns-            boolean noLockExists = !result.getColumnDefinitions().contains("client");-            if (result.getBool("[applied]") || noLockExists) {-                log.info("Lock released for {} by client {} at: {}", lockName, clientId, System.currentTimeMillis());-                return true;-            }--            String clientReleasingLock = result.getString("client");-            if (!clientReleasingLock.equals(clientId)) {-                if (isRetryAfterWriteTimeout) {-                    log.info("Released lock for client {} in retry attempt after WriteTimeoutException", clientReleasingLock);+            if (result != null) {+                // if a row doesn't exist then cassandra doesn't send back any columns+                boolean noLockExists = !result.getColumnDefinitions().contains("client");

How does this play with the null check above now? Is noLockExists ever going to be true?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 ALTER TABLE status DROP waste_of_space;-ALTER TABLE status ADD waste_of_space text;+ALTER TABLE status ADD waste_of_space varchar;

Why did this change?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 public void shouldThrowExceptionForBootstrapWithMissingSemiColon() throws Except         Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap_missing_semicolon"));          //when-        MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+        MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);          //then-        cluster.connect(TEST_KEYSPACE);+        //throws exception

Could we catch the throwable and assert it's type instead of nothing? This has highlighted that it's not obvious which functional call the exception was previously being thrown from

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

-ALTER TABLE status ADD waste_of_space double;+ALTER TABLE status ADD waste_of_space text;

Why did this change?

sethishi

comment created time in 8 days

Pull request review commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

 package uk.sky.cqlmigrate; -import com.datastax.driver.core.Cluster;+import com.datastax.oss.driver.api.core.CqlSession;+import com.datastax.oss.driver.api.core.session.Session;++import java.net.InetSocketAddress;+import java.util.List;+import java.util.stream.Collectors;+import java.util.stream.Stream;  public class CassandraClusterFactory {      /**-     * Creates an instance of cassandra {@link Cluster} based on the provided configuration+     * Creates an instance of cassandra {@link Session} based on the provided configuration      *-     * @param hosts Addresses of the nodes to add as contact points (as described in-     *              {@link Cluster.Builder#addContactPoint}).-     * @param port The port to use to connect to the Cassandra hosts.+     * @param hosts    Addresses of the nodes to add as contact points (as described in+     *                 {@link CqlSession}).+     * @param port     The port to use to connect to the Cassandra hosts.      * @param username the username to use to login to Cassandra hosts.      * @param password the password corresponding to {@code username}.-     *-     * @return a configured Cluster+     * @return a configured Session      */-    public static Cluster createCluster(String[] hosts, int port, String username, String password) {-        Cluster.Builder builder = Cluster.builder()-                .addContactPoints(hosts)-                .withPort(port);+    public static Session createCluster(String[] hosts, int port, String username, String password) {+        List<InetSocketAddress> cassandraHosts = Stream.of(hosts)+                .map(host -> new InetSocketAddress(host, port))+                .collect(Collectors.toList());          if (username != null && password != null) {-            builder = builder.withCredentials(username, password);+            return CqlSession.builder()+                    .addContactPoints(cassandraHosts)+                    .withAuthCredentials(username, password).build();         } -        return builder.build();+        return CqlSession.builder().addContactPoints(cassandraHosts).build();

This class has changed from providing a Cluster to a Session, and then isn't actually used by code elsewhere in the project now; some questions

  • Should it be renamed to CassandraSessionFactory?
  • Should it return a CqlSession rather than a Session?
  • Is it needed? Should other places in main code be using this, or should this class be removed as it's not currently used?
sethishi

comment created time in 8 days

pull request commentsky-uk/cqlmigrate

Update cqlmigrate to use cassandra driver 4.x

Rebased on top of latest master; original branch pushed to update-cassandra-driver-4.x-with-simulacron-ORIGINAL. Comparing this branch with that shows only whitespace changes at time of push.

sethishi

comment created time in 8 days

push eventsky-uk/cqlmigrate

Shilpa Sethi & Stelios Anastasakis

commit sha 3a9ae58d59aa28a58b478334d69c37b19d9d499d

Update Cassandra driver to 4.11.1 Updated method signature for migrate to include datacenter Updated test data as driver no longer allows updating incompatible datatype of columns

view details

David Heath

commit sha 475f0a65c1f20dc9e0d2136adb345b17c263139d

Handle NPE in CassandraLockingMechanism

view details

Shilpa Sethi

commit sha c111bc7f45fcaed3279c0f915d23bd41261eab02

Reduce logging for build to pass on Travis

view details

push time in 8 days