Wednesday, 2 August 2017

Steps to implement a search in S&P With AEM

Set up
  • Hosting: Hosting the search tool.
  • Indexing: Indexing the data.
  • Index frequency : Configuring the index.       

Configure
  • Metadata - Used for faceting, sorting, ranking, relevance.
  • Breadcrumbs - for a better navigation.
  • Pagination - for a better navigation.
  • Recent Searches - Search assist.
  • Search suggestions(Did you mean) - Search assist.
  • Auto complete - Search assist.

Fine-tune
  • Promotion: Normal, Based on search result, we can promote.
  • Dictionaries : Configure for better results.
  • Banners : any organizational promotion.
  • Redirects : Send user to a page.

Read more here

Related 
Steps to implement any search technology
AEM Dispatcher, why it  is needed?
AEM Desktop App
Figure out the best search technology or tool
Steps to implement search in Solr
Quality of Search - fine tuning search implementation

How do we select the best search tool for the site?

There are many search engines, some are open source and some are paid. The trend in search technology shows an affinity towards cognitive and artificial intelligence now.

  • Solr
  • Elasticsearch
  • Google Search Appliance (GSA)
  • Oracle Endeca
  • Microsoft FAST
  • Attivio
  • Sinequa
  • Coveo
  • IBM Watson
  • Amazon CloudSearch
  • SharePoint Search
  • HP Autonomy

The common Big data used with searches are:
  • Apache Hadoop
  • Cloudera
  • Hortonworks
Search Trend Evolution
Keyword Search -> Semantic -> Contextual -> Cognitive -> (Human Brain)

Enterprise search now uses natural language processing and machine learning which dramatically improve the results relevancy and completeness.
Here cognitive search is based on Artificial Intelligence.

Which search is the best?

I always suggest the one that meets your needs in your environment and requirements.

Process of identifying the needs

Auditing the current system
Understand the current system and new requirements.
OS & Systems
Think of operating system to be hosted. On-Premise Vs Cloud Vs Hybrid
Dev tools
Think on the tools to develop while search is getting revamped
Repositories
Think about content repository and ensure product upgrade history is clean.
Security
Are there any new security levels to be added?
Content
Think on the content when search system upgrades are in place
Users
External Vs internal? The behavior & type of users.

Understand any data
Search solutions must connect to and ingest data from a wide variety of source. For e.g.: Data types ranging from images, video, audio, and machine data such as from internet-of-things (IoT) devices.

Scale to handle big
Now the data is in peta bytes which resides in distributed architectures.

The migration process
Provide transparent, solid experience to the end users, Educate the users about new system.

Install base and revenue history
A proven stream of revenue generated by customer adoption of its solution and installation market presence.

Cross-domain standalone solution, Employ AI technologies
The solution is self-sufficient, general-purpose cross domain one now. Understand and organize data, predict, improve relevancy, and automatically tune the relevancy of results over time using AI Techniques.

Allow developers to customize search applications
Currently search vendors provide SDKs, Apis, and in some cases visual design tools to customize search to a maximum extend.

List out your semi finalists then finalists. Then rematch the requirement with finalists and decide the winner.

Read More
Steps to implement any search technology
AEM Dispatcher, why it  is needed?
AEM Desktop App
Figure out the best search technology or tool
Steps to implement search in Solr
Quality of Search - fine tuning search implementation

Quality of search - Fine tuning the search

There are many search engines, some are open source and some are paid. The trend in search technology shows an affinity towards cognitive and artificial intelligence now.
  • Solr
  • Elasticsearch
  • Google Search Appliance (GSA)
  • Oracle Endeca
  • Microsoft FAST
  • Attivio
  • Sinequa
  • Coveo
  • IBM Watson
  • Amazon CloudSearch
  • SharePoint Search
  • HP Autonomy
The common Big data used with searches are:
  • Apache Hadoop
  • Cloudera
  • Hortonworks
Quality of search(QoS)
Let us see how we can make a search effective. QoS is a term used with search implementation fine tuning. Once a search is implemented, next steps are to ensure the search performance. There are many ways we can do the fine tuning.

Query cleaning: 
Before search hits the index, ensure the query is cleaned and aligned to your index.

Add analytics:
Ensure you are using analytics hand in hand with search, which will definitely improve the effectiveness of search.

Rules: 
Rules like ranking, relevance etc can be adjusted to fine tune the search results.

Proper metadata mapping:
Re-verify the metadata mapping for index to ensure everything is done perfectly.

Dictionaries, Synonyms:
Some times we miss to configure dictionaries which will restrict the user from synonym search. Words with similar meaning does not come in result set at all. Confirm you have configured them well.

Excluded words:
Ensure the excluded word list is configured so that none of the unwanted queries fetch the result.

Restrictions:
There are cases where we have to skip some pages from search results. Ensure this restrictions are working as expected.

Once all above conditions are met, next step is to verify the search functionality ourselves to confirm the 'recent searches', 'search suggestion', 'auto complete'/ 'type ahead', 'null search results' are functioning as expected.

Finally don't miss to check the reports like Terms report, null search report, search requests, index reports, analytics reports,  crawl reports, content request reports.

Usually search technology is treated as 'once implement then ignore'. My recommendation is keep improving the configurations by constantly watching the reports often.

Read More
Steps to implement any search technology
AEM Dispatcher, why it  is needed?
AEM Desktop App
Figure out the best search technology or tool
Steps to implement search in Solr
Quality of Search - fine tuning search implementation