Build faster Search APIs with WebSockets
Before the establishment of REST APIs in 2000, “there was no standard on how to design an API or use one.” SOAP did exist, but it was complex to use in the eyes of the community. People have a distaste for XML, I’ve witnessed this firsthand trying to push GopherSauce (also used XML). Since the 2000s, new standards were developed as well. Amongst those is GraphQL. GraphQL is explicit, in the sense that you define “which kind of object you can request and what fields it has.” GraphQL is a REST API with bells and whistles, as it still uses the same protocol as REST, which is HTTP. This is why I’m still reluctant to switch to GraphQL, as it does not provide any benefits I want. The benefit being performance. Whether it’s GraphQL or REST, there is still a request going to the server, and the client is waiting for a response to render. What if I told you that you can return response data as the server is processing it?
WebSockets
WebSockets enable real-time communication between the client and server. Talks of WebSockets began in 2008, ( I wonder why Facebook didn’t incorporate it with GraphQL) initial support came with Google Chrome 4 in 2010. Nowadays, most browsers support the protocol. WebSockets use cases include chat and multiplayer games. As your Go idiot, I thought of performing search with WebSockets.
Puttin’ it to work
In this section, I will build a theoretical Go HTTP handler. This handler will upgrade an HTTP request to use web sockets. The handler will search a directory’s files for the specified regex pattern. I’ll be using Gorilla WebSockets for my implementation. The following code will be the gist of the handler, it’ll upgrade the HTTP request to use a web socket.
If you noticed above, I’m invoking InitQuery as a Go routine. This moves the actual search to another thread in the system. InitQuery will be responsible for dispatching the search on the actual files. A picture is worth 1000 words, so here is an implementation of function InitQuery :
InitSearch will spawn a Go routine for each file. The function named Process will search for matches in the specified files, if a match is found, it will lock the mutex, write to the connection and return. With this architecture, a result is returned as the system discovers it. However, WebSockets aren’t magical, it has its own set of limitations too. Recently, I had a WebSocket application that closed unexpectedly. The problem turned out to be sending non UTF-8 characters to the client. As a solution, I used the function ToValidUTF8 from the bytes package. A terrible solution, as it replaced unrecognized characters with nothing, literally nothing.
Where do we go from there?
WebSockets are powerful, in the sense that it brings a transport protocol to the browser. It also enables Web applications to get results in realtime. Now pairing WebSockets with Goroutines makes for a quick and powerful web server. Let’s say I was to implement the code above without webSockets, I’d have to wait for all the Goroutines to finish, as I can only write a response once. I’ll still benefit from parallel processing, but my code will grow in complexity trying to track the completion of the request. I plan on implementing this search style in TheStrukture IDE.
If you enjoyed the insight in this post, subscribe to my Gumroad course for more hacks I picked up along my Go Journey : https://thestrukture.gumroad.com/l/strukture_live_training
