profile
viewpoint
Moquette moquette-io Italy http://moquette.io Moquette is lightweight MQTT Java broker

moquette-io/moquette 1880

Java MQTT lightweight broker

moquette-io/netty-mqtt5-codec 22

Netty codec implementation for MQTT3 and MQTT5

issue commentmoquette-io/moquette

Does this broker has bridge support? if yes can you tell me how to configure and achive it .

I just started a issue-thread with the mosquitto team .. ;-) might be they can help here. As soon I have any news I will udpate this issue and let you know the outcome.

Many thanks for your time

keshavck

comment created time in 6 hours

issue commentmoquette-io/moquette

Does this broker has bridge support? if yes can you tell me how to configure and achive it .

Ah, indeed, that's a totally different MQTT version number! I guess mosquitto does some extra things when a connection is made using that version number. If they have documentation specifying exactly how that works, it may be possible to also implement it in Moquette...

keshavck

comment created time in 6 hours

issue commentmoquette-io/moquette

Does this broker has bridge support? if yes can you tell me how to configure and achive it .

Hello @hylkevds thanks for the update and I agree with you. I did also not find any hint about a special Bridge Protocol. But here is what I got in wireshard.

By default mosquitto send a frame like below: Mosquitto-default

This is what you posted in your prev. reply.

Now when I configure mosquitto as a bridge I see the following in wireshark:

Mosquitto-Bridge

As you see the protocol headre is set to 0x84 and not as expected as 0x40.

That's why I think that mosquitto does something special here. For me it looks like that mosquittor is doings somthing which is not in the MQTT spec.

Not sure if all works fix it would be better to mask the protocol byte the "0x7F= to delete the MSB before checking the protocol version. To get moquette working with a mosquitto bridge .. might we can introduce the mask-function as a configuration... just an idea.. it is not perfect I know. What doy ou think?

The best would be that I discsuss this with the mosquitto community first adn post thge outcome of the discussion here. Does this make sense to you ?

Thanks again for your great support Regards

Reinhold

keshavck

comment created time in 6 hours

issue commentmoquette-io/moquette

Does this broker has bridge support? if yes can you tell me how to configure and achive it .

There is no "MQTT-bridge protocoll" that is standardised in the MQTT spec.

Here is a typical MQTT connect packet:

0000   00 00 03 04 00 06 00 00 00 00 00 00 00 00 08 00   ................
0010   45 00 00 75 d4 0e 40 00 40 06 68 72 7f 00 00 01   E..u..@.@.hr....
0020   7f 00 00 01 eb 28 07 5c f8 64 11 b9 34 0f 33 16   .....(.\.d..4.3.
0030   80 18 02 00 fe 69 00 00 01 01 08 0a a5 e6 46 2c   .....i........F,
0040   a5 e6 44 f8 10 3f 00 04 4d 51 54 54 04 00 00 1e   ..D..?..MQTT....
0050   00 33 46 52 4f 53 54 2d 4d 51 54 54 2d 42 75 73   .3FROST-MQTT-Bus
0060   2d 32 36 62 62 37 35 34 32 2d 36 38 33 32 2d 34   -26bb7542-6832-4
0070   37 63 37 2d 38 33 39 35 2d 62 36 66 63 30 64 30   7c7-8395-b6fc0d0
0080   36 65 33 39 38                                    6e398

The actual packet content is:

0040               10 3f 00 04 4d 51 54 54 04 00 00 1e       .?..MQTT....
0050   00 33 46 52 4f 53 54 2d 4d 51 54 54 2d 42 75 73   .3FROST-MQTT-Bus
0060   2d 32 36 62 62 37 35 34 32 2d 36 38 33 32 2d 34   -26bb7542-6832-4
0070   37 63 37 2d 38 33 39 35 2d 62 36 66 63 30 64 30   7c7-8395-b6fc0d0
0080   36 65 33 39 38                                    6e398

Comparing with the spec: 0x10: CONNECT Packet fixed header 0x3f: Length of the rest of the MQTT packet 0x00: Length MSB (0) 0x04: Length LSB (4) 0x4d: M 0x51: Q 0x54: T 0x54: T 0x04: Version 4 (is 3.1.1)

So if your packet starts with 0x84 it's not MQTT... Hence my suspicion of SSL... What did you get in wireshark?

keshavck

comment created time in 7 hours

issue commentmoquette-io/moquette

Does this broker has bridge support? if yes can you tell me how to configure and achive it .

Many thanks for the hint, I will double check.

As far as I see it is not an connection problem. I see in Wireshark the the mosquitto-bridge brokers sends an MQTT-Protocoll Version 3.1.1 which is encoded as 0x84. So the connection ewas established. I tested it where SLL was NOT used on boths sides..

So my guess is that there is some kind of magic MQTT-bridge protocoll .. unfortunatelly I did not find any details about siuch an protocoll ( like specifgication) Might I need to take a closer look to the source code for both borker implementation.

At least I found this in the moquette sources of MQTTConnection.java

void processConnect(MqttConnectMessage msg) {
        MqttConnectPayload payload = msg.payload();
        String clientId = payload.clientIdentifier();
        final String username = payload.userName();
        LOG.trace("Processing CONNECT message. CId: {} username: {}", clientId, username);

        if (isNotProtocolVersion(msg, MqttVersion.MQTT_3_1) && isNotProtocolVersion(msg, MqttVersion.MQTT_3_1_1)) {
            LOG.warn("MQTT protocol version is not valid. CId: {}", clientId);
            abortConnection(CONNECTION_REFUSED_UNACCEPTABLE_PROTOCOL_VERSION);
            return;
        }

So it look like that the received protocol header byte is 08x then connection is rejected.

I will check the logging information again because the moquette is running as an embedded broker.. I need some time to figure it out asking some experts who did the originbal coding.

Many thanks for your support. I will update the isseu when I found someting Cheers

keshavck

comment created time in 11 hours

issue commentmoquette-io/moquette

Does this broker has bridge support? if yes can you tell me how to configure and achive it .

I've been confused by an incorrect MQTT protocol header a while back, and after a lot of debugging I found out that the side that initiated the connection was trying to use SSL. Since the SSL handshake comes before the MQTT header, the server side that was looking for an MQTT header was confused by the SSL header and thus throw a fit. So, make sure your configuration is set up for SSL on both sides, or on neither side!

keshavck

comment created time in 12 hours

issue commentmoquette-io/moquette

Does this broker has bridge support? if yes can you tell me how to configure and achive it .

Hello all, I have the same question.

In a pure Mosquitto environment there is no problem between the communication of two brokers where one broker is configures as MQTT-Bridge. (See below)

TEST PASSED :.. MQTT Explorer <--> mosquitto broker <---> mosquitto broker bridge <--> MQTT Explorer

this works fine

When I change my environment like below ( just replace one mosquite borker with mosqutte) it fails

FAILED : MQTT Explorer <--> mosquette broker <---> mosquitto broker bridge <--> MQTT Explorer

The Mosquitto-Bridge broker is not able to connect to the mosuette.

Analyses of the problem shows the following:

mosquitto tries to establish a connection and sends the the protocoll version for MQTT 3.1.1 . This is normally encodeded in the MQTT header protocol header as 0x04 . In case of an MQTT bridge config the mosquitto-bridge-broker sends 0x84.

As you see the highest bit ( 2^7) is set to 1.

I think this is the reason why the mosquette rejects the connect because 0x04 is expected.

Unfortunately I did not find any definition of the bit 2^7 in the protocoll version bye.

Call to action:

Would be great is someone has a solution for this --> like special config setting in moquette

OR let let me know if I should open an issue for this

Many thanks in advance Cheers Reinhold

keshavck

comment created time in 12 hours

PR opened moquette-io/moquette

Moquette broker refactor

h2-mvstore on the classpath caused me some issues running the broker from Quarkus. Workaround:

    <dependency>
      <groupId>io.moquette</groupId>
      <artifactId>moquette-broker</artifactId>
      <version>0.15</version>
      <exclusions>
        <!-- to avoid a CHECK2 NoSuchFieldError -->
        <exclusion>
          <groupId>com.h2database</groupId>
          <artifactId>h2-mvstore</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

So I dove into the moquette sources a bit. Initially it seemed like bumping the h2 version would resolve this, but on deeper look there are some pretty nice interfaces present in moquette (IQueueRepository, ISubscriptionsRepository, IRetainedRepository) that seem like they could enable moquette to make a clean divide between the broker and the H2 implementations:

This is a large body of work I don't expect to be merged, but I wanted to get an idea of the maintainers' openness to such a move, and any alternative approaches they would take to get there. I'll also note that this work is also incomplete- the goal would be the removal of all H2 dependencies from moquette-broker, moving the current H2 persistence backends to their own module.

    <dependency>
      <groupId>io.moquette</groupId>
      <artifactId>moquette-broker</artifactId>
      <version>0.15</version>
    </dependency>

    <!-- H2 implementation classes -->
    <dependency>
      <groupId>io.moquette</groupId>
      <artifactId>moquette-repository-h2</artifactId>
      <version>0.15</version>
    </dependency>

    <!-- Redis implementation classes -->
    <dependency>
      <groupId>io.moquette</groupId>
      <artifactId>moquette-repository-redis</artifactId>
      <version>0.15</version>
    </dependency>

    <!-- JSR-107 implementation classes -->
    <dependency>
      <groupId>io.moquette</groupId>
      <artifactId>moquette-repository-jcache</artifactId>
      <version>0.15</version>
    </dependency>

There are numerous advantages to this from an embedder's perspective, particularly in an EE context.

Branch builds and tests pass, although ErrorProne is disabled- moquette does not build with vanilla JDK11 / mvn clean package with that present. The README is outdated, referring to gradle wrappers that seemingly no longer exist.

+702 -398

0 comment

99 changed files

pr created time in 18 hours

startedmoquette-io/moquette

started time in 19 hours

startedmoquette-io/moquette

started time in 2 days

issue commentmoquette-io/moquette

Dropping session queues leaks netty ByteBufs, resulting in OOM

I've not found the time to properly look through it all, work is always a bit crazy near the end of the year. Three more weeks till my Holidays. Should find the time then :)

For normal publishes "dropping" the message on a full queue (and not sending an ACK) should be the best solution. That should automatically cause the load to reduce, since those messages will be re-sent, but in the mean time they take up an in-flight-slot on the client, meaning that client can't send as much at the same time.

I also realised that in my test it was caused by a thread using the internalPublish of the broker. We should probably change that to return true when the internal publish was queued successfully, and false if it was dropped due to a full queue. That way the thread that does the internal publish can delay for a few milliseconds and retry.

jbutler

comment created time in 2 days

issue commentmoquette-io/moquette

Dropping session queues leaks netty ByteBufs, resulting in OOM

Hi @hylkevds I should have fixed the first point of your latest test. Now on PUB qos1 and PUB qos2 (first step) if the target session queue is full then no ack are sent, PUBACK(QoS1) or PUBREC(QoS2). It keeps a cache of packetIds for source clientID that failed, this table is used to to understand when the a publish of a specific packetId is completed and so go forward with the remaining publish duties.

So please, could you give another try to that PR, so that I can check with next bugs? Many thanks for your effort in making this project progressing.

jbutler

comment created time in 2 days

startedmoquette-io/moquette

started time in 3 days

startedmoquette-io/moquette

started time in 3 days

startedmoquette-io/moquette

started time in 4 days

startedmoquette-io/moquette

started time in 11 days

startedmoquette-io/moquette

started time in 12 days

startedmoquette-io/moquette

started time in 13 days

startedmoquette-io/moquette

started time in 13 days

startedmoquette-io/moquette

started time in 13 days

startedmoquette-io/moquette

started time in 15 days

issue commentmoquette-io/moquette

java.lang.NoSuchMethodError: No static method encodeHexString([B)Ljava/lang/String; in class Lorg/apache/commons/codec/binary/Hex; or its super classes

This piece of code is used inside moquette's ResourceAuthenticator. And internally this invokes Hex.encodeHexString which is not available in android api 27's runtime. Even if we use higher version of commons.codec in android project, during runtime it uses the Hex class present inside android's runtime which is a lower version. @amitkj-stack I hope you are also facing this issue in android rite?

amitkj-stack

comment created time in 19 days

issue commentmoquette-io/moquette

java.lang.NoSuchMethodError: No static method encodeHexString([B)Ljava/lang/String; in class Lorg/apache/commons/codec/binary/Hex; or its super classes

I am also facing the same issue. And this issue occurs in android API 27 because the ResourceAuthenticator's checkValid uses DigestUtils.sha256Hex(...) and this is not available in API 27's commons codec. @amitkj-stack by any chance did you find any solution to this problem?

amitkj-stack

comment created time in 20 days

more