Hello

I am building a small service where we need to get the user details from LDAP. I was able to search the users successfully when I run the code in my desktop however once the code is deployed in AWS lambda, search is not working, from the logs I can see that the ldap connection is good and there are no errors. its just that search is not returning the user info. Please advise.

import { Client, SearchOptions, createClient } from 'ldapjs';

import { Injectable } from '@nestjs/common';
import { Config} from '@shared/config/app.config';
import { LogService} from '@shared/services/log/log.service';
import { SecretManagerService} from '@shared/services/secret/secret_manager.service';

@Injectable()
export class DSLdap {
	private client: Client = null;

	constructor(private readonly logService: LogService, private readonly config: Config, private readonly secretService: SecretManagerService) {}

	async config(): Promise<void> {
		let username: string;
		let password: string;

		const secretString: string = await this.secretService.getSecretValue(this.dsConfig.ldap.secretKeyName);
		if (secretString) {
			this.logService.info('LDAP secret retrieved successfully.');
			({ username, password } = JSON.parse(secretString));
		}
		this.logService.info('Connecting to LDAP server...');
		this.client = createClient({
			url: this.config.ldap.url,
			bindCredentials: '',
			tlsOptions: {
				rejectUnauthorized: false
			},
			log: this.logService
		});
		this.logService.info('Connected to LDAP server...');

		await new Promise((resolve, reject) => {
			this.logService.info('Binding to LDAP server...');
			this.client.bind(username, password, (err) => {
				if (err) {
					reject(err);
				}
				this.logService.info('Bound to LDAP server...');
				return resolve(true);
			});

			this.client.on('error', (error) => {
				this.logService.error(error);
				return reject(error);
			});
		});
	}

	async searchByEmail(emailAddress: string): Promise<any> {
		const opts: SearchOptions = {
			filter: `(userPrincipalName=${emailAddress})`,
			scope: 'sub'
		};
		const entries: any = [];
		const referrals: any = [];
		return new Promise((resolve, reject) => {
			this.client.on('error', (error) => {
				this.logService.error(error);
				return reject(error);
			});

			this.client.search(this.config.ldap.baseDn, opts, (err, resp) => {
				if (err) {
					reject(err);
				}
				resp.on('searchEntry', (entry) => {
					this.logService.info('Entry ' + entry);
					entries.push(entry.object);
				});
				resp.on('searchReference', (referral) => {
					this.logService.info('Referral ' + referral);
					referrals.push(referral.uris);
				});

				resp.on('error', (error) => {
					this.logService.error(error);
					return reject(error);
				});

				resp.on('end', (result) => {
					this.logService.info('Result ' + result);
					return resolve({
						entries: entries,
						referrals: referrals
					});
				});
			});
		});
	}
}

Closing this as it sounds like you have found your issue and solution.

Reminder: we do not support transpiled versions of the library.

Hello Everyone,

I was able to make it work. The problem is with webpack. When we bundle the application, webpack config uses managle and also rename the functions and classes name to smaller character in order to save space which affects the event emitter. In my case the function name SearchEntry is replaced with character 'y' due to mangle and this causes issue when it reaches this line of code " msg.constructor.name" in manageCallback shown below where it expects the constructor name as SearchEntry which will then be changed to searchEntry and send that as an event. However in my case the constructor name is coming as 'y' and the event is emitted as 'y' which is why it is not getting any results. In order to keep the function name and class name I used below config for my webconfig.

From client.js

 function messageCallback (msg) {
    if (timer) { clearTimeout(timer) }

    log.trace({ msg: msg ? msg.json : null }, 'response received')

    if (expect === 'abandon') { return sendResult('end', null) }

    if (msg instanceof SearchEntry || msg instanceof SearchReference) {
      let event = msg.constructor.name
      event = event[0].toLowerCase() + event.slice(1)
      return sendResult(event, msg)
    } else {
      tracker.remove(message.messageID)
      // Potentially mark client as idle
      self._updateIdle()

      if (msg instanceof LDAPResult) {
        if (expect.indexOf(msg.status) === -1) {
          return sendResult('error', errors.getError(msg))
        }
        return sendResult('end', msg)
      } else if (msg instanceof Error) {
        return sendResult('error', msg)
      } else {
        return sendResult('error', new errors.ProtocolError(msg.type))
      }
    }
  }

Webpack fonfig:

minimizer: [ new TerserPlugin({ terserOptions: { mangle: true, keep_classnames: true, keep_fnames: true, sourceMap: true }, parallel: true }) ]

Same issue here. Using Directus.io (which uses knex) on DigitalOcean and MySQL 8. I hope this gets fixed soon.

@DanielSamsonraj

I tried adding the certificate to the nodejs env and it works perfectly fine in local but still no luck in lambda. No error nothing...just get empty response.

Yep... this issue actually yells for support to give regexp style matching to allow more complex load patterns.

Current config doesn't allow to distinguish for example .generated.js and .d.ts from desired .js and .ts files

loadExtensions:['.js'] seems like will fix my issue! Thank you :)

I do think that knex trying to load .d.ts files is a bug, it should try to load either .ts or .js files.

@elhigu Thanks, yes, eventually I found the problem, it's not a problem of knex, I can put my problematic code here to give people a bad case:

      res.status(200).json(transferJson2);
      await session.commitTransaction();

Here res is Express response object. So I commit the database transaction too later, when I move the below line to the above, I get the correct result.

@Kematia I added loadExtensions:['.js'] to knexfile.js and your repo worked without producing an error.

migrations: {
          tableName: "knex_migrations",
          directory: "./dist/migrations",
          loadExtensions:['.js']
      },

And I had to install ts-node as well. Not exactly sure if this is a bug or expected behavior

I can look into this.

Any guidance on where I should get started will be appreciated.

I've set up a repo to reproduce the error: https://github.com/Kematia/knex-loads-types

Even when used with toString, it does not work, e.g. when you have a custom format with an array of items. It seems, that the algorithm finding a match in the rv._sensitive set is not working then.

Environment

Knex version: 0.95.6 Database + version: SQLite 3.31.0 OS: macOS 11.4

Bug

1. Explain what kind of behaviour you are getting and how you think it should do

When using dropColumn, any existing foreign key constraints are copied over to the temp table. This causes it to attempt to create a foreign key constraint for a field that isn't included in the create table sql.

2. Error message

CREATE TABLE "_knex_temp_alter709" (`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL, `regular_column` varchar(255) NULL, CONSTRAINT related_fk_column_foreign FOREIGN KEY (`fk_column`) REFERENCES `related` (`id`) ON DELETE SET NULL) - SQLITE_ERROR: unknown column "fk_column" in foreign key definition

3. Reduced test code, for example in https://npm.runkit.com/knex or if it needs real database connection to MySQL or PostgreSQL, then single file example which initializes needed data and demonstrates the problem.

await knex.schema.createTable('table_a', table => {
  table.increments();
});

await knex.schema.createTable('table_b', table => {
  table.increments();
  table.integer('fk_column').references('id').inTable('table_a');
});

await knex.schema.alterTable('table_b', table => {
  table.dropColumn('fk_column');
});

After digging into this issue I have some unfortunate findings. Firstly that the code which sets UV_THREADPOOL_SIZE doesn't work, and will actually set a UV_THREADPOOL_SIZE of NaN since this.driver.poolMax is undefined. I assume the correct value should be this.pool.max.

However, this doesn't really matter since UV_THREADPOOL_SIZE is ignored unless its set before execution. At least on Ubuntu 20.04.

It seems like the only thing we really can do is make it clear that Oracle users should set their UV_THREADPOOL_SIZE appropriately.

This would be a great feature, please do accept it 👍