Introducing Sketchboard Content Search
We are happy to introduce a new search with content matches. It strives to show relevant findings with highlighted terms.
For many years, Sketchboard indexed board content and showed the search result with board thumbnails. Based on your visual memory, you were able to choose the correct match.
However, the actual search matches were not shown, what kind of context the term has, and what was the exact keyword used. It was good, most of the time, but it could have been better. Especially now that teams have created more online whiteboards with Sketchboard over time.
Search result with shape links
When Sketchboard shows the search result, on the first row, it shows the board name. After that, it shows diagram/shape element text fragments that match the search, with a direct URL to the shape on your whiteboard.
When you click the board title, Sketchboard opens the board as usual. In case, you directly select the matched shape by clicking the shape link in the search result. Sketchboard opens the board, and navigates to the element by centering it on your browser.
Shape links also work within a board. When you click a shape link inside a board, Sketchboard navigates to the shape. If you have more than one shape id on a link, Sketchboard navigates to referred shapes and centers the board on them.
URL format for a shape link is:
https://sketchboard.me/[board_id]/[shape_id1,shape_id2]
The shape URL can be used directly on any web page to target a certain shape or shapes.
<a href='https://sketchboard.me/[board_id]/[shape_id]'/>A direct shape link</a>
Sketchboard doesn’t allow you to access a shape link easily other than using the search. In addition to search, you can access the shape link by copying the shape to the clipboard. Please send us feedback to request a more straightforward way to get shape id(s) quickly!
Fuzzy Search
By default, content search uses a fuzzy search. Fuzzy search matches within a specified edit distance. E.g., when searching
team
Can lead to finding terms such as:
teams
term
Fuzzy search is sometimes too broad. You can change the search to an explicit match by using quotes on a search term(s).
"content marketing"
Preview Viewer for Search Result
Sometimes the search result is not good enough. Then you can use Preview Viewer to have a close look for the board thumbnail. Preview Viewer allows you to navigate through the matched board thumbnails.
Indexing Frequency
Sketchboard indexes your online whiteboard content after a few minutes you have updated the board. It is not exactly realtime, but quite close. Now it is easier to find the correct board later and directly navigate to the desired shape.
Technology Behind The Search - Bleve Search
We decided to go with the new search implementation with Bleve search, which is a Go language library.
Why we went with Bleve search instead of ElasticSearch?
- Go language allows ease of deployment
- Fits nicely into our development and production stack
- Ease of development, configuration, and updates
- Relatively fast
- No need to use JVM
Elastic search is probably the top-notch solution on the search engine frontier, but Bleve is mostly good enough. At least for now. Time will tell if the performance and search results are good enough when using Bleve.
Production load
Bleve search cons: sometimes, the startup takes a lot of time. It can be over 2 minutes for a store of the size of 15 GB using Bolt database, https://github.com/boltdb/bolt.
Sketchboard Updates on using Bleve May 18th, 2020
Sketchboard search engine was changed to Bleve Search and Bolt database a few months ago. Here are some experiences of the usage.
The Bolt database file kept growing, and startup time to load Bolt database became a bottleneck, and also queries took sometimes a very long time, mainly when the search database was not used for some time. Probably due to Bolt database is a single file and becomes easily massive, and can cause problems to access different positions in the file when not used for a while, perhaps related to GC (garbage collection).
We did some research, and Bleve Search nowadays provides a Scorch solution to store indexed search results. It wasn’t trivial to find out the documentation on how to change Bolt to Scorch, but in the end, it was a simple one-liner to make the switch.
By default, Bleve uses Bolt as an index database when you are using a new Bleve database creation.
index, err := bleve.New(bleveDBDir, indexMapping)
If you want to use Scorch, all you need to do is to change that line with the following construction.
index, err := bleve.NewUsing(
bleveDBDir,
indexMapping,
scorch.Name,
scorch.Name,
nil,
)
There is no tool to convert your existing Bolt database to Scorch, and you need to reindex your search database. We switched Bolt to Scorch on May 16th, 2020, and the first experiences of the difference are impressive.
- Scorch database is about 1/4 of size
- The database is opened in few hundreds of milliseconds
- Searches are fast at any time
We are thrilled with Bleve search and their new Scorch implementation, huge props for the Bleve team for this achievement!
Test Bleve and Scorch search performance and results on Sketchboard online whiteboard. Sign up and create a board with text content.