< Search Platform < Weekly Updates
Search Platform/Weekly Updates/2023-09-22
Summary
We're moving from developing features to polishing then on the Search Update Pipeline. And testing the various requirements for our production deployment. Deployment of the Search Update Pipeline and migration of all wikis to the new pipeline will happen next quarter.
We are working through optimizing the performance of our improvement to multilingual zero-results rate.
What we've accomplished
Search Update Pipeline
- Ongoing testing of Flink / Zookeeper integration - https://phabricator.wikimedia.org/T342149
- Ongoing refactoring to extract common configuration and graph factories
- Alignement of WDQS Streaming Updater with current best practices - https://phabricator.wikimedia.org/T326914
- Add a way to filter input events per wiki - https://phabricator.wikimedia.org/T345634
- Consider dropping support for java8 - https://phabricator.wikimedia.org/T346015
Improve multilingual zero-results rate
- First draft of implementing the analysis improvemnts a
Operations
- Search incident during the datacenter switch. Performance was decreased enough so that we dropped search queries. The system recovered mostly on its own once caches were warmed up. We're learning things and we will be better prepared for the next one. - https://wikitech.wikimedia.org/wiki/Incidents/2023-09-20_Elasticsearch_unavailable
Misc
- Replaced the use of "whitelist" with "allowlist" in the WDQS code base (this follows a similar change to our puppet code - https://phabricator.wikimedia.org/T343856) - https://phabricator.wikimedia.org/T344284
This article is issued from Wikimedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.