my nixos configuration
my emacs configuration
Terminal app that spams RGB signals to fix stuck pixels.
solutions for https://adventofcode.com/2017
Build highly concurrent, distributed, and resilient message-driven applications on the JVM
jsravn/akka-kubernetes-tests 0
Basic tests that show Akka Cluster and friends work on Kubernetes
Ansible modules - these modules ship with ansible
AssertJ is a library of assertions similar to fest-assert but providing a richer set of assertions.
Customised fork of cluster-autoscaler to support machine-controller-manager
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void shouldNotLoadAnyOfTheCqlFilesIfThereAreDuplicateCqlFilenames() throw //when try {- MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+ MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths); fail("Should have died"); } catch (RuntimeException e) { // nada } //then- KeyspaceMetadata keyspaceMetadata = cluster.getMetadata().getKeyspace(TEST_KEYSPACE);- assertThat(keyspaceMetadata).as("should not have made any schema changes").isNull();+ Optional<KeyspaceMetadata> keyspaceMetadata = session.getMetadata().getKeyspace(TEST_KEYSPACE);+ assertThatThrownBy(keyspaceMetadata::get).as("should not have made any schema changes").isInstanceOf(NoSuchElementException.class);
Can just assert that keyspaceMetadata is empty?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public boolean release(String clientId) throws CannotReleaseLockException { ResultSet resultSet = session.execute(deleteLockQuery.bind(lockName, clientId)); Row result = resultSet.one(); - // if a row doesn't exist then cassandra doesn't send back any columns- boolean noLockExists = !result.getColumnDefinitions().contains("client");- if (result.getBool("[applied]") || noLockExists) {- log.info("Lock released for {} by client {} at: {}", lockName, clientId, System.currentTimeMillis());- return true;- }-- String clientReleasingLock = result.getString("client");- if (!clientReleasingLock.equals(clientId)) {- if (isRetryAfterWriteTimeout) {- log.info("Released lock for client {} in retry attempt after WriteTimeoutException", clientReleasingLock);+ if (result != null) {+ // if a row doesn't exist then cassandra doesn't send back any columns
Remember how this was found/proven? The only test we have is mocked and curious how this was discovered
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public boolean acquire(String clientId) throws CannotAcquireLockException { ResultSet resultSet = session.execute(insertLockQuery.bind(lockName, clientId)); Row currentLock = resultSet.one(); // we could already hold the lock and not be aware if a previous acquire had a writetimeout as a timeout is not a failure in cassandra- if (currentLock.getBool("[applied]") || clientId.equals(currentLock.getString("client"))) {+ //TODO this needs to be tested+ if (currentLock != null && (currentLock.getBoolean("[applied]") || clientId.equals(currentLock.getString("client")))) {
Still needs testing?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void rowInsertedWithMessageDigestHashingAlgorithmIsSameAsGuavaSha1Hashing //when schemaUpdates.add(- filename,- Paths.get(cqlResource.toURI())+ filename,+ Paths.get(cqlResource.toURI()) ); //then final String guavaSha1Hash = Resources.asByteSource(cqlResource).hash(Hashing.sha1()).toString(); final ResultSet resultSet = session.execute("SELECT * from " + SCHEMA_UPDATES_TABLE);- assertThat(resultSet.all())- .hasOnlyOneElementSatisfying(row -> {- assertThat(row.getString("filename"))+ assertThat(resultSet.all().size()).isEqualTo(1);+ resultSet.forEach(row ->+ {+ assertThat(row.getString("filename")) .isEqualTo(filename);- assertThat(row.getString("checksum"))+ assertThat(row.getString("checksum")) .isEqualTo(guavaSha1Hash);- });+ }); }
any reason to change this out of curiosity? Don't mind either way.
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void tearDown() { } @AfterClass- public static void tearDownClass() throws Exception {+ public static void tearDownClass() { EmbeddedCassandraServerHelper.cleanEmbeddedCassandra(); Uninterruptibles.sleepUninterruptibly(100, TimeUnit.MILLISECONDS); } @Test- public void schemaUpdatesTableShouldBeCreatedIfNotExists() throws Exception {+ public void schemaUpdatesTableShouldBeCreatedIfNotExists() { //given- cluster.connect("system").execute("CREATE KEYSPACE " + TEST_KEYSPACE + " WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1 };");- Session session = cluster.connect(TEST_KEYSPACE); SessionContext sessionContext = new SessionContext(session, ConsistencyLevel.ALL, ConsistencyLevel.ALL, clusterHealth); SchemaUpdates schemaUpdates = new SchemaUpdates(sessionContext, TEST_KEYSPACE); //when schemaUpdates.initialise(); //then- KeyspaceMetadata keyspaceMetadata = cluster.getMetadata().getKeyspace(TEST_KEYSPACE);+ KeyspaceMetadata keyspaceMetadata = session.getMetadata().getKeyspace(TEST_KEYSPACE).get(); assertThat(keyspaceMetadata.getTable(SCHEMA_UPDATES_TABLE)).as("table should have been created").isNotNull();
can check optional is empty? if the test fails the .get() with throw an exception rather than fail this assertion.
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void shouldNotRemoveLockAfterMigrationFailed() throws Exception { //when try {- MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+ MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths); } catch (RuntimeException ignore) { } //then- ResultSet resultSet = session.execute("SELECT * FROM cqlmigrate.locks WHERE name = ?", LOCK_NAME);- assertThat(resultSet.isExhausted()).as("Is lock released").isFalse();+ ResultSet resultSet = session.execute(newInstance("SELECT * FROM cqlmigrate.locks WHERE name = ?", LOCK_NAME));+ assertThat(resultSet.one()).as("Is lock released").isNotNull(); } @Test public void shouldRemoveLockAfterMigrationFailedIfUnlockOnFailureIsSetToTrue() throws Exception { //given CqlMigrator migrator = new CqlMigratorImpl(CqlMigratorConfig.builder()- .withLockConfig(CassandraLockConfig.builder().unlockOnFailure().build())- .withReadConsistencyLevel(ConsistencyLevel.ALL)- .withWriteConsistencyLevel(ConsistencyLevel.ALL)- .build(), new SessionContextFactory());+ .withLockConfig(CassandraLockConfig.builder().unlockOnFailure().build())+ .withReadConsistencyLevel(ConsistencyLevel.ALL)+ .withWriteConsistencyLevel(ConsistencyLevel.ALL)+ .build(), new SessionContextFactory()); Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap_missing_semicolon")); //when try {- migrator.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+ migrator.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths); } catch (RuntimeException ignore) { } //then- ResultSet resultSet = session.execute("SELECT * FROM cqlmigrate.locks WHERE name = ?", LOCK_NAME);- assertThat(resultSet.isExhausted()).as("Is lock released").isTrue();+ ResultSet resultSet = session.execute(newInstance("SELECT * FROM cqlmigrate.locks WHERE name = ?", LOCK_NAME));+ assertThat(resultSet.one()).as("Is lock released").isNull(); } @Test public void shouldRetryWhenAcquiringLockIfNotInitiallyAvailable() throws Exception { //given- session.execute("INSERT INTO cqlmigrate.locks (name, client) VALUES (?, ?)", LOCK_NAME, UUID.randomUUID().toString());+ session.execute(newInstance("INSERT INTO cqlmigrate.locks (name, client) VALUES (?, ?)", LOCK_NAME, UUID.randomUUID().toString())); Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap")); //when Future<?> future = executorService.submit(() -> MIGRATOR.migrate(- CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths));+ CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths)); session.execute("TRUNCATE cqlmigrate.locks"); Thread.sleep(1000); future.get(); //then try {- cluster.connect(TEST_KEYSPACE);+ session.getMetadata().getKeyspace(TEST_KEYSPACE);
Are these checking the same thing? getKeyspace() returns an optional which could still be empty if the bootstrap hasn't been applied?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
private static final String KEYSPACE = "locker"; private static String TABLE_NAME = KEYSPACE + ".lock_testing"; - private Cluster cluster;- private Session session;+ private static Server server = Server.builder().build();+ private ClusterSpec cluster = ClusterSpec.builder().build();+ private BoundCluster bCluster;
Now that this test uses simulacron rather than real cassandra, could the @ignore be removed? Can it run in pipelines now?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void shouldObtainLockToInsertRecord() throws InterruptedException { } private void updateCounter(int updatedCounter) {- Statement updateCounterQuery = new SimpleStatement(String.format("UPDATE %s SET counter = %d WHERE id = 'lock-tester'", TABLE_NAME, updatedCounter)).setConsistencyLevel(ConsistencyLevel.QUORUM);+ Statement updateCounterQuery = SimpleStatement.newInstance(String.format("UPDATE %s SET counter = %d WHERE id = 'lock-tester'", TABLE_NAME, updatedCounter)).setConsistencyLevel(ConsistencyLevel.QUORUM); session.execute(updateCounterQuery); } private int readCurrentCounter() {- Statement readCounterQuery = new SimpleStatement(String.format("SELECT counter from %s WHERE id = 'lock-tester'", TABLE_NAME)).setConsistencyLevel(ConsistencyLevel.QUORUM);+ Statement readCounterQuery = SimpleStatement.newInstance(String.format("SELECT counter from %s WHERE id = 'lock-tester'", TABLE_NAME)).setConsistencyLevel(ConsistencyLevel.QUORUM); final ResultSet currentRecord = session.execute(readCounterQuery); return currentRecord.one().getInt("counter"); } - private Cluster createCluster() {- QueryOptions queryOptions = new QueryOptions();- queryOptions.setConsistencyLevel(ConsistencyLevel.LOCAL_QUORUM);-- SocketOptions socketOptions = new SocketOptions();- socketOptions.setReadTimeoutMillis(1000);-- return Cluster.builder()- .addContactPoints(CASSANDRA_HOST)- .withPort(CASSANDRA_PORT)- .withQueryOptions(queryOptions)- .withSocketOptions(socketOptions)- .build();- }
We no longer specify timeouts in this test. Were they needed?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void shouldThrowExceptionForInvalidBootstrap() throws Exception { Collection<Path> cqlPaths = singletonList(cqlPath); //when- MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+ MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths); //then- cluster.connect(TEST_KEYSPACE);+ //throws exception
Could we catch the throwable and assert it's type instead of nothing? This has highlighted that it's not obvious which functional call the exception was previously being thrown from
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void schemaUpdatesTableShouldNotBeCreatedIfExists() throws Exception { fail("Expected " + SCHEMA_UPDATES_TABLE + " table creation to be attempted only once."); } //then- KeyspaceMetadata keyspaceMetadata = cluster.getMetadata().getKeyspace(TEST_KEYSPACE);+ KeyspaceMetadata keyspaceMetadata = session.getMetadata().getKeyspace(TEST_KEYSPACE).get(); assertThat(keyspaceMetadata.getTable(SCHEMA_UPDATES_TABLE)).as("table should have been created").isNotNull();
can check optional is empty? if the test fails the .get() with throw an exception rather than fail this assertion.
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void migrate(Session session, String keyspace, Collection<Path> directori /** * {@inheritDoc} */- public void clean(String[] hosts, int port, String username, String password, String keyspace) {- try (Cluster cluster = CassandraClusterFactory.createCluster(hosts, port, username, password);- Session session = cluster.connect()) {- this.clean(session, keyspace);+ public void clean(String[] hosts, String localDC, int port, String username, String password, String keyspace) {++ List<InetSocketAddress> cassandraHosts = Stream.of(hosts).map(host -> new InetSocketAddress(host, port)).collect(Collectors.toList());++ try (CqlSession cqlSession = CqlSession.builder()
(mentioned earlier) no longer using the factory to create the session; deliberate?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void shouldThrowExceptionWhenMultipleBootstrap() throws Exception { Collection<Path> cqlPaths = asList(getResourcePath("cql_bootstrap"), getResourcePath("cql_bootstrap_duplicate")); //when- MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+ MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths); //then- cluster.connect(TEST_KEYSPACE);+ //throws exception
Could we catch the throwable and assert it's type instead of nothing? This has highlighted that it's not obvious which functional call the exception was previously being thrown from
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public static void main(String[] args) { .collect(Collectors.toList()); CqlMigratorFactory.create(CassandraLockConfig.builder().build())- .migrate(hosts.split(","), port == null ? 9042 : Integer.parseInt(port), username, password, keyspace, directories);+ .migrate(hosts.split(","), localDC, port == null ? 9042 : Integer.parseInt(port), username, password, keyspace, directories); } /** * {@inheritDoc} */- public void migrate(String[] hosts, int port, String username, String password, String keyspace, Collection<Path> directories) {- try (Cluster cluster = CassandraClusterFactory.createCluster(hosts, port, username, password);- Session session = cluster.connect()) {- this.migrate(session, keyspace, directories);+ public void migrate(String[] hosts, String localDC, int port, String username, String password, String keyspace, Collection<Path> directories) {+ List<InetSocketAddress> cassandraHosts = Stream.of(hosts).map(host -> new InetSocketAddress(host, port)).collect(Collectors.toList());++ try (CqlSession cqlSession = CqlSession.builder()
(mentioned earlier) no longer using the factory to create the session; deliberate?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public class ClusterHealthTest { private static final int defaultStartingPort = PortScavenger.getFreePort();+ private static final Server server = Server.builder().build(); - private ClusterHealth clusterHealth;+ private final ClusterSpec cluster = ClusterSpec.builder().build(); - private static Server server = Server.builder().build();- private ClusterSpec cluster = ClusterSpec.builder().build();+ private ClusterHealth clusterHealth; private BoundCluster bCluster;-- private Cluster realCluster;+ private CqlSession realCluster; @Before public void setUp() throws UnknownHostException { DataCenterSpec dc = cluster.addDataCenter().withName("DC1").withCassandraVersion("3.11").build();- dc.addNode().withAddress( new InetSocketAddress(Inet4Address.getByAddress(new byte[] {127, 0, 0, 1}), defaultStartingPort)).build();+ dc.addNode().withAddress(new InetSocketAddress(Inet4Address.getByAddress(new byte[]{127, 0, 0, 1}), defaultStartingPort)).build(); dc.addNode().withPeerInfo("host_id", UUID.randomUUID()).build(); bCluster = server.register(cluster); bCluster.prime(when("select cluster_name from system.local where key = 'local'") .then(rows().row("cluster_name", "0").build())); - realCluster = CassandraClusterFactory.createCluster(new String[]{"localhost"}, defaultStartingPort, null, null);- realCluster.connect();+ realCluster = CqlSession.builder()+ .addContactPoint(new InetSocketAddress(Inet4Address.getByAddress(new byte[]{127, 0, 0, 1}), defaultStartingPort))+ .withLocalDatacenter(dc.getName())+ .build();
Same question about not using the factory for creating cluster/session?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void shouldDeleteLockWhenReleasingLock() throws Exception { //then assertThat(isLockReleased).isTrue(); List<QueryLog> queryLogs = cluster.getLogs().getQueryLogs().stream()- .filter(queryLog -> queryLog.getFrame().message.toString().contains(PREPARE_DELETE_QUERY))- .collect(Collectors.toList());+ .filter(queryLog -> queryLog.getFrame().message.toString().contains(PREPARE_DELETE_QUERY))+ .collect(Collectors.toList()); assertThat(queryLogs.size()).isEqualTo(1); } @Test- public void shouldSuccessfullyReleaseLockWhenNoLockFound() throws Exception {+ public void shouldSuccessfullyReleaseLockWhenNoLockFound() { cluster.prime(primeDeleteQuery(LOCK_KEYSPACE, CLIENT_ID, false)); lockingMechanism.init(); //when
What is different between this test and shouldReturnTrueIfNoLockExists from a functional perspective?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
} void initialise() {- Session session = sessionContext.getSession();- session.execute(new SimpleStatement("USE " + keyspace + ";").setConsistencyLevel(sessionContext.getReadConsistencyLevel()));- TableMetadata schemaUpdateTableMetadata = session.getCluster().getMetadata().getKeyspace(keyspace).getTable(SCHEMA_UPDATES_TABLE);+ CqlSession session = sessionContext.getSession();+ session.execute(SimpleStatement.newInstance("USE " + keyspace + ";").setConsistencyLevel(sessionContext.getReadConsistencyLevel()));+ TableMetadata schemaUpdateTableMetadata = session.getMetadata().getKeyspace(keyspace).get().getTable(SCHEMA_UPDATES_TABLE).orElse(null);
Since we have optionals could we use them? getKeyspace(XXX).map(k -> k.getTable()).orElse(null)? While functionally the keyspace should exist, the .get() call is unchecked and could result in future issues.
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
void bootstrap() { Session session = sessionContext.getSession();- KeyspaceMetadata keyspaceMetadata = session.getCluster().getMetadata().getKeyspace(keyspace);+ KeyspaceMetadata keyspaceMetadata = session.getMetadata().getKeyspace(keyspace).orElse(null); if (keyspaceMetadata == null) {
Could we make use of optional isPresent rather than mapping to null and immediately checking for null?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void shouldMigrateSchemaIfLockCanBeAcquired() throws Exception { Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap")); //when- MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+ MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths); //then try {- cluster.connect(TEST_KEYSPACE);+ session.getMetadata().getKeyspace(TEST_KEYSPACE);
Are these checking the same thing? getKeyspace() returns an optional which could still be empty if the bootstrap hasn't been applied?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
CassandraLockConfig lockConfig = CassandraLockConfig.builder() .withLockKeyspace("cqlmigrate") .build(); -// Create a Cassandra session-Cluster cluster = Cluster.builder.addContactPoint("localhost").build();-Session session = cluster.connect();+// Create a Cassandra session for cassandra driver 4.x+CqlSession session = CqlSession.builder()+ .addContactPoints(cassandraHosts)+ .withLocalDatacenter("datacenter1")+ .withAuthProvider(new ProgrammaticPlainTextAuthProvider("username", "password"))+ .build();
nitpick, but the import in this code snippet is no longer accurate
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void shouldRunTheBootstrapCqlIfKeyspaceDoesNotExist() throws Exception { Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap")); //when- MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+ MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths); //then try {- cluster.connect(TEST_KEYSPACE);+ session.getMetadata().getKeyspace(TEST_KEYSPACE);
Are these checking the same thing? getKeyspace() returns an optional which could still be empty if the bootstrap hasn't been applied?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
package uk.sky.cqlmigrate; -import com.datastax.driver.core.Cluster;-import com.datastax.driver.core.Host;+import com.datastax.oss.driver.api.core.metadata.Node;+import com.datastax.oss.driver.api.core.metadata.NodeState;+import com.datastax.oss.driver.api.core.session.Session; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import uk.sky.cqlmigrate.exception.ClusterUnhealthyException; import java.net.InetAddress; import java.util.List;-import java.util.Set;+import java.util.Map;+import java.util.UUID; import java.util.stream.Collectors; class ClusterHealth { private static final Logger log = LoggerFactory.getLogger(ClusterHealth.class); - private final Cluster cluster;+ private final Session session; - ClusterHealth(Cluster cluster) {- this.cluster = cluster;+ ClusterHealth(Session session) {+ this.session = session; } void check() throws ClusterUnhealthyException { - log.debug("Checking cluster health");+ log.debug("Checking session health"); - Set<Host> allHosts = cluster.getMetadata().getAllHosts();+ Map<UUID, Node> nodes = session.getMetadata().getNodes(); - List<InetAddress> unhealthyHosts = allHosts- .stream()- .filter(host -> !host.isUp())- .map(Host::getAddress)+ List<InetAddress> unhealthyHosts = nodes.values().stream()+ .filter(node -> node.getState().equals(NodeState.DOWN))+ .map(node -> node.getBroadcastAddress().get().getAddress()) .collect(Collectors.toList()); if (!unhealthyHosts.isEmpty()) { throw new ClusterUnhealthyException("Cluster not healthy, the following hosts are down: " + unhealthyHosts); } - log.debug("All hosts healthy: {}", allHosts);+ log.debug("All hosts healthy: {}", nodes.entrySet());
This will log a bunch of UUIDs which may not be that useful; should it log the values() instead?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
* and manual intervention is required to cleanup and fix the issue. * * @param hosts Comma separated list of cassandra hosts+ * @param localDC Local Datacenter name to be provided to create a session for cassandra driver 4.x.x * @param port Native transport port for the above cassandra nodes * @param username Username for Cassandra's internal authentication using PasswordAuthenticator * (if using AllowAllAuthenticator, can be set to any value) * @param password Password for Cassandra's internal authentication using PasswordAuthenticator * (if using AllowAllAuthenticator, can be set to any value) * @param keyspace Keyspace name for which the schema migration needs to be applied * @param directories Comma separated list of directory paths containing the cql statements for the schema change- * @throws ClusterUnhealthyException if any nodes are down or the schema is not in agreement before running migration- * @throws CannotAcquireLockException if any of the queries to acquire lock fail or- * {@link CassandraLockConfig.CassandraLockConfigBuilder#withTimeout(Duration)}- * is reached before lock can be acquired.- * @throws CannotReleaseLockException if any of the queries to release lock fail- * @throws IllegalArgumentException if any file types other than .cql are found- * @throws IllegalStateException if cql file has changed after migration has been run- * @throws com.datastax.driver.core.exceptions.DriverException if any of the migration queries fails+ * @throws ClusterUnhealthyException if any nodes are down or the schema is not in agreement before running migration+ * @throws CannotAcquireLockException if any of the queries to acquire lock fail or+ * {@link CassandraLockConfig.CassandraLockConfigBuilder#withTimeout(Duration)}+ * is reached before lock can be acquired.+ * @throws CannotReleaseLockException if any of the queries to release lock fail+ * @throws IllegalArgumentException if any file types other than .cql are found+ * @throws IllegalStateException if cql file has changed after migration has been run+ * @throws com.datastax.oss.driver.api.core.DriverException if any of the migration queries fails
Since we're updating javadoc (albein whitespace) could we remove the fully qualified name and import instead?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public boolean release(String clientId) throws CannotReleaseLockException { ResultSet resultSet = session.execute(deleteLockQuery.bind(lockName, clientId)); Row result = resultSet.one(); - // if a row doesn't exist then cassandra doesn't send back any columns- boolean noLockExists = !result.getColumnDefinitions().contains("client");- if (result.getBool("[applied]") || noLockExists) {- log.info("Lock released for {} by client {} at: {}", lockName, clientId, System.currentTimeMillis());- return true;- }-- String clientReleasingLock = result.getString("client");- if (!clientReleasingLock.equals(clientId)) {- if (isRetryAfterWriteTimeout) {- log.info("Released lock for client {} in retry attempt after WriteTimeoutException", clientReleasingLock);+ if (result != null) {+ // if a row doesn't exist then cassandra doesn't send back any columns+ boolean noLockExists = !result.getColumnDefinitions().contains("client");
How does this play with the null check above now? Is noLockExists ever going to be true?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
ALTER TABLE status DROP waste_of_space;-ALTER TABLE status ADD waste_of_space text;+ALTER TABLE status ADD waste_of_space varchar;
Why did this change?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
public void shouldThrowExceptionForBootstrapWithMissingSemiColon() throws Except Collection<Path> cqlPaths = singletonList(getResourcePath("cql_bootstrap_missing_semicolon")); //when- MIGRATOR.migrate(CASSANDRA_HOSTS, binaryPort, username, password, TEST_KEYSPACE, cqlPaths);+ MIGRATOR.migrate(CASSANDRA_HOSTS, LOCAL_DC, binaryPort, username, password, TEST_KEYSPACE, cqlPaths); //then- cluster.connect(TEST_KEYSPACE);+ //throws exception
Could we catch the throwable and assert it's type instead of nothing? This has highlighted that it's not obvious which functional call the exception was previously being thrown from
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
-ALTER TABLE status ADD waste_of_space double;+ALTER TABLE status ADD waste_of_space text;
Why did this change?
comment created time in 8 days
Pull request review commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
package uk.sky.cqlmigrate; -import com.datastax.driver.core.Cluster;+import com.datastax.oss.driver.api.core.CqlSession;+import com.datastax.oss.driver.api.core.session.Session;++import java.net.InetSocketAddress;+import java.util.List;+import java.util.stream.Collectors;+import java.util.stream.Stream; public class CassandraClusterFactory { /**- * Creates an instance of cassandra {@link Cluster} based on the provided configuration+ * Creates an instance of cassandra {@link Session} based on the provided configuration *- * @param hosts Addresses of the nodes to add as contact points (as described in- * {@link Cluster.Builder#addContactPoint}).- * @param port The port to use to connect to the Cassandra hosts.+ * @param hosts Addresses of the nodes to add as contact points (as described in+ * {@link CqlSession}).+ * @param port The port to use to connect to the Cassandra hosts. * @param username the username to use to login to Cassandra hosts. * @param password the password corresponding to {@code username}.- *- * @return a configured Cluster+ * @return a configured Session */- public static Cluster createCluster(String[] hosts, int port, String username, String password) {- Cluster.Builder builder = Cluster.builder()- .addContactPoints(hosts)- .withPort(port);+ public static Session createCluster(String[] hosts, int port, String username, String password) {+ List<InetSocketAddress> cassandraHosts = Stream.of(hosts)+ .map(host -> new InetSocketAddress(host, port))+ .collect(Collectors.toList()); if (username != null && password != null) {- builder = builder.withCredentials(username, password);+ return CqlSession.builder()+ .addContactPoints(cassandraHosts)+ .withAuthCredentials(username, password).build(); } - return builder.build();+ return CqlSession.builder().addContactPoints(cassandraHosts).build();
This class has changed from providing a Cluster to a Session, and then isn't actually used by code elsewhere in the project now; some questions
- Should it be renamed to CassandraSessionFactory?
- Should it return a CqlSession rather than a Session?
- Is it needed? Should other places in main code be using this, or should this class be removed as it's not currently used?
comment created time in 8 days
pull request commentsky-uk/cqlmigrate
Update cqlmigrate to use cassandra driver 4.x
Rebased on top of latest master; original branch pushed to update-cassandra-driver-4.x-with-simulacron-ORIGINAL. Comparing this branch with that shows only whitespace changes at time of push.
comment created time in 8 days
push eventsky-uk/cqlmigrate
Shilpa Sethi & Stelios Anastasakis
commit sha 3a9ae58d59aa28a58b478334d69c37b19d9d499d
Update Cassandra driver to 4.11.1 Updated method signature for migrate to include datacenter Updated test data as driver no longer allows updating incompatible datatype of columns
commit sha 475f0a65c1f20dc9e0d2136adb345b17c263139d
Handle NPE in CassandraLockingMechanism
commit sha c111bc7f45fcaed3279c0f915d23bd41261eab02
Reduce logging for build to pass on Travis
push time in 8 days