Browser integration with WebSockets, Kafka and KSQLDB

Tony Kambourakis, Principal Architect, IBM
Jamin Adey, Principal Architect — Digital, Fidelity Life Assurance
Tarun Sharma, Azure Enterprise Solutions Architect, IBM

Overview

A popular browser integration style used in developing browser-based business applications is synchronous HTTP-based APIs. The browser application client code, be it JavaScript or TypeScript, either requests data or submits data to one of various endpoints, typically mapped to a server-side application, a Backend For Frontend (BFF) and a microservice. In turn the microservice may retrieve data from or commit data to a database or publish an event/message to an event/messaging system such as IBM MQ, AWS SQS or Apache Kafka.

HTTP-based API integration

Alternatively, a chat style application tends to use an asynchronous form of integration such as WebSocket. These allow for both client and server to initiate the transmission of data from either end, without the need for a response. Similarly, dashboard style applications can utilise a WebSocket to receive a continuous stream of data to update their views.

In this article we look to explore a solution in combining WebSockets with Kafka for a business application that would typically have used an HTTP API based integration style. This provides the browser application client with a single integration style to both submit and receive data, without the need for polling for updates from the server-side, which does not only waste resources, but introduces increased latency; a) by having to wait for next poll frequency to occur before obtaining the result, which may have already been processed b) by not having the added overhead of recreating HTTPS handshake, and requiring the full set of HTTP headers to be sent with each request.

We’ll use Azure SignalR and Confluent Cloud (an Apache Kafka Platform-as-a-Service provider) with KSQLDB as examples.

WebSocket Connections

The solution comprises the browser application client utilising the SignalR SDK to establish a WebSocket connection with the Azure SignalR service to send and receive data. The data can be structured in the form of commands or events. In the example below, the browser application client follows a micro-frontend (MFE) design with separate bounded contexts provided by an Accounts MFE and a Transactions MFE. Each MFE has its own WebSocket connection to its associated SignalR Hub, one for each bounded context.

The goal is for each MFE to run independently along with its own backing services. The only thing that should be shared between the MFEs when moving from one to another (usually within a parent hosting web app) is some kind of unique session/object identifier passed at the UI layer (often in the URL which forms an agreed contract between MFEs). This means the backing services of each MFE (i.e. within its bounded context) requires its own up-to-date knowledge of the context, rather than having shared state (either frontend e.g. redux or backend) across them which would tightly couple them and remove the benefits that modular, independently deployable Micro-frontend and Microservices patterns bring. Using a technology such as Kafka, especially leveraging Confluent’s KSQLDB streaming capability, highly simplifies publishing and subscribing to events across bounded contexts to enable loosely-coupled choreography patterns.

Azure SignalR Hubs are there to group a set of common messages, much like a Topic in Apache Kafka. With MFEs scoped to a bounded context, and with their own SignalR hub, then any messages will be guaranteed to only be part of that bounded context, which a) ensures encapsulation b) introduces granular auth control as connecting to a hub for a particular MFE can now be authorised, with only permitted users to be able to use the particular MFE’s services (5).

The Accounts MFE presents a list of the customer’s accounts with their respective balances that it retrieves using a “RetrieveAccountsCmd” command. The Accounts Azure Function receives the “RetrieveAccountsCmd” command and performs a pull query against the associated Accounts KSQLDB materialised view to return the accounts for that customer. The Accounts Function sends the account list to the browser application client as an independent event — “AccountListUpdatedEvent”. Any updates to the account summary list will trigger the Account Function, through the KSQLDB materialised view’s underlying backed topic.

The triggered Account Function would then send the Account List Updated Event with the revised account list to the client.

WebSocket-based integration with Azure SignalR

When a browser client connects to the Azure SignalR service it is assigned a unique session identifier. The downstream system receiving events sent to SignalR may send messages to clients using either the session identifier, authenticated user identifier or a group identifier. To avoid polluting the Kafka topics and streams with any SignalR specific identifiers (session identifier), the customer identifier is used.

When the Account Function is trigged by Account List Updated events from Kafka, it receives events for all customers. The function will send the event to the appropriate browser instance (via SignalR socket session) by addressing it to the customer identifier.

For Kafka, Azure functions uses the built-in integration capability “Kafka Trigger” to listen to any changes in the underlying Kafka topic and receive all the new messages or events published in the topic. Similarly, the Azure function uses the built-in integration capability “Output binding” for publishing a message or event to a topic (3). Only configuration (no coding) in terms of broker details, topic name, protocol etc. is required to publish or subscribe to the Kafka topic. An alternative approach is to use the KSQLDB SDK to publish to a stream.

Concurrent Browser Sessions

A key benefit of using WebSocket connections is the ability to send updates to multiple browsers concurrently. This avoids having multiple browsers send polling requests into your services to check if there is new or changed data. However, there may be a business requirement to restrict access by a user from multiple browsers/tabs concurrently.

The Azure SignalR service offers a publish/subscribe mechanism for easily broadcasting messages to multiple clients using Groups (1). When a connection is established with Azure SignalR a client is given a Connection ID. A client may be subscribed to the group by adding its Connection ID to the group (6).

Expanding on the accounts example the following scenario includes an already established connection from a browser (Browser Instance #1) and the subsequent connection of a second browser instance by the same user (Browser Instance #2). In this case the requirement is to allow only a single browser instance to interact with the user’s data at any one time. A new browser instance will be able to connect, any previously connected browser instance will be disconnected.

Concurrent browser connections

The example begins after the user has authenticated. In our application, we need to support two scenarios: customer self-service and staff assisting a customer. This means that there will be two possible identity types who have authenticated, the customer and a staff member.

When the customer logs in, their customer ID will be minted as a user identifier claim in the JWT. For a staff member, only the staff ID will be minted as a user identifier in the JWT and not the customer ID, as a staff member can perform assisted actions for any of their given customers. For customers therefore, the customer ID in the JWT can be trusted to perform authorisation that any actions are scoped to that customer’s business and what they’re permitted to do with it. For staff, the customer ID will need to be passed as a parameter, and therefore, the operations invoked will require ensuring that customer is indeed one the staff member is authorised to perform specific actions for.

The browser has received a JWT containing the authenticated user identifier. Browser Instance #1 has already connected and has been interacting with the system. The user launches Browser Instance #2 and authenticates.

1. The Accounts MFE sends a connection request to the Socket Negotiate Azure Function.
2. The Socket Negotiate function initiates a connection with the Azure SignalR service that returns a connection URL and Access Token that is returned to the browser.
3. The Accounts MFE establishes a connection with the Azure SignalR Account Hub, using the connection URL and Access Token.
4. The Accounts MFE sends the AddCustomerSessionMappingCmd command to the Account Hub. The micro UI starts a timeout timer that it will use to confirm the system has registered its new connection identifier by it receiving a SessionMappingUpdated message.
5. The Account Hub forwards the AddCustomerSessionMappingCmd to the Update Customer Session Mapping function.
6. The Update Customer Session Mapping function adds Browser Instance #2’s Connection ID to the SignalR group that has the group name set to the user identifier (if the group does not exist, a new group is created). The function utilises the Azure SignalR SDK that internally pushes the group update to the SignalR service.
7. The Update Customer Session Mapping function broadcasts the RemoveDuplicateSession message to the group, excluding the Connection ID (2) it just added for the Browser Instance #2. Any browser connections within the group will receive this message and close their connection to the SignalR service, optionally informing the user that another session has been established and restricting them from proceeding.
8. (a) The Browser Instance #1 MFE receives the RemoveDuplicateSession message and closes the SignalR connection. It can present to the user a message that another session has been established, restricting the user from continuing with the application. (b) The Update Customer Session Mapping function broadcasts the SessionMappingUpdated message to the group that now only includes the Browser Instance #2.
9. The Browser Instance #2 Accounts MFE receives the SessionMappingUpdated message and stops the timeout timer, confirming its connection has been registered.
10. Accounts MFE sends RetrieveAccountsCmd command to Account Hub.
11. Accounts Hub forwards RetrieveAccountsCmd to the Accounts Aure function that in turn reads from the Accounts KSQLDB Materialised View.
12. The Accounts function sends the Account List Updated event to the Account Hub, addressed to the group name.
13. The Account Hub forwards the Account List Updated event to the Accounts MFE.

By setting the group name equal to the user identifier the various functions can source data from Kafka/KSQLDB using business identifiers rather than SignalR specific identifiers such as the Connection ID. In the step where the Accounts function is triggered by an update to the Account List, the KSQLDB/Kafka topic will contain the user identifier that the change is related to. The Accounts function will then use that user identifier as the group name it sends the Account List Updated event without needing to know the browser’s SignalR connection identifier.

Handling Disconnections

If the web socket connection is momentarily broken due to a loss of network connection SignalR will attempt to reconnect and invoke the “reconnecting” connection lifetime event. This could trigger the browser application client to provide an alert to the user and block them from continuing to make updates via the UI.

Upon reconnection SignalR assigns the browser client a new Connection ID, replacing the previous Connection ID in the group with the new one. The replacement of the Connection ID happens transparently to the browser application client code and as such there is no need to send the AddCustomerSessionMappingCmd command.

Error Handling and Correlation

Since this type of solution relies on asynchronous integration (Fire & Forget or Notifications), error handling needs a different approach. In an API-based synchronous integration, the calling thread would have the status of the request success or failure, it is easy to perform error handling processing. In an asynchronous integration, the publication of a message or event may have a consequence unknown to the publisher, there is a need for some mechanism to communicate any downstream error (if and when happen) to the publisher.

For KsqlDB, “KSQL_Processing_Log” provides details of all the stream processing activities performed on Topic, Stream, and Tables including any error such as schema validation errors (4). The data in this log can be used to identify the error that occurred by transforming the error information appropriately and storing it in another topic which then can trigger an error handling function with the details and cause of the error. The error handling Azure function then can communicate to the particular UI Client via the SignalR service.

There may be in some cases a need for a correlation of the error or (even a successful response) so that the error can be stitched (correlated) to the original message or event to be played back to the corresponding UI Client. In scenarios where a retry publication request is made this “correlationID” can be used to ignore all other responses however this warrants the error or success response contains the “correlationID” which is the unique MessageID of the request.

References

1. Working with Groups in SignalR, Guide to the API, Microsoft
2. HubClientsExtensions.GroupExcept Method, Microsoft.AspNetCore.SignalR, Microsoft
3. Azure Functions triggers and bindings concepts, Microsoft
4. KSQL Process Log, KSQLDB, Confluent
5. ASP.NET SignalR Hubs API Guide — JavaScript Client, Microsoft
6. Managing SignalR ConnectionIds (or why you shouldn’t), Kevin W. Griffin