You are currently viewing Web Search Evaluator – A complete authoritative guide

Web Search Evaluator – A complete authoritative guide


This post may contain affiliate links, which means we may receive a commission, at no extra cost to you, if you make a purchase through a link. Please see our full affiliate disclosure for further information.

Web Search Evaluator

This post will tell you anything and everything you need to know about the web search evaluator job.

I will lay threadbare all the inside details about this job that is not available online. I am an active evaluator and hence I know everything about this job.

If you are a stay at home mama, a college student, an introvert, jobless or anyone who is looking to work online, then this is by far, the best online job you will ever find in the entire world wide web.

And it is much more stable and consistent than survey sites.

Hey, I made $2007 working as a web search evaluator aka search engine evaluator being an Indian although we are paid half of what Americans or Europeans are paid.

Find out what it is and how to work as one –

What is Web Search Evaluator aka Search Engine Evaluation?

Web Search Evaluator Job Description

Web Search Evaluation aka Search Engine evaluation is a process of evaluating and rating websites and web-pages based on the General Guidelines given by Google or Microsoft. It involves finding out the quality of websites and how relevant or helpful a result is for the queries issued by user.

Look at this image below –

This is what search engine evaluator does

A user (like you and me) issued a query world war ii in his/her mobile device.  And he/she got the result as shown above. (Other results below are truncated due to lack of space)

The result is pretty helpful to a user issuing the query world war ii

Go ahead, type something similar if you feel like.

Now, in search engine evaluation, we have to rate how relevant or useful a particular result is to a given query.

Results could be web-pages, websites, YouTube videos, apps, pdf files, songs, and so on. We also got to evaluate the quality of web-pages or websites.

So in short, a web search evaluation aka search engine evaluation is a process of evaluating websites and web-pages based on the parameters outlined in General Guidelines of Google ( or Microsoft) and also finding out how relevant or useful a particular result is for a given query.

Due to NDA I signed with them, I cannot disclose any specific tasks that we do inside.

Terminologies simplified 

There are different terms used for people who work as a search engine evaluator or web search evaluator or Google Rater. They are also known as internet analyst, raters and so on. They all mean the same thing.

Here are four different terms with which they are known –

  • Web Search Evaluator
  • Search Engine Evaluator
  • Personalized Internet Assessor
  • Internet Safety Analyst
  • Internet Analyst
  • Google Rater
  • Rater

I will be using these terms interchangeably here. They will all mean the same thing. Please don’t get confused.

Different companies uses different terms for them

Appen (previously Leapforce) likes to call them internet analyst.

Raterlabs likes to call them internet analyst

Lionbridge calls it personalized internet assessors. They also refer to it as raters or as internet safety analyst

But they all mean the same thing – Google Rater, search engine evaluator aka web search evaluator.

What do Search Engine Evaluators do?

A search engine evaluator evaluates web-pages and websites and judges its quality. However, a search engine evaluators main work is to rate the relevance and helpfulness of a result to a query issued by a user.

How to work as search engine evaluators?

The process involves applying to the companies mentioned below and waiting for them to get back to you. It can take anywhere from few days to few months. If they don’t get back to you within 6 months, you may re-apply again.

But till its 6 months, do not re-apply with different email accounts because in your first application itself, your IP address would be captured. Re-applying again with different emails would only send you back to the waiting queue.

Initial Requirements Checklist.

  • Have in-depth, up-to-date familiarity with social culture, media, and web culture
  • Excellent comprehension and written communication skills in English
  • Broad range of interests, with specific areas of expertise a plus
  • University degree or equivalent experience (degrees in-progress are acceptable). Advanced degrees a plus. (In my opinion, it is  not really required. Just apply for the positions available).
  • Excellent web research skills and analytical abilities.
  • Ability to work independently with minimal supervision
  • Possess a high-speed internet connection (Cable Modem, Fiber, DSL, etc.)
  • Use of an Android phone version 4.1 or higher, Windows phone version 8.1 or higher, or an iPhone version 4s or higher

Plus, you have to be Googly enough :). I mean, you have to be comfortable using Google to research stuffs, look for things and be comfortable looking around a bit.

Where to apply and how to apply?

Three companies are hiring at the moment –

If you want an in-depth comparison of the 4 evaluator companies, read this article HERE

Applying to these companies is pretty straight forward. Just click on the links above, you are taken to an application page, you just need to fill in your basic details, upload CV and bamm ! You are done.

Lionbridge application process is a bit lengthy though. You need to fill in all your details, academic, work experience and others, and upload CV as well.

{ NB – Leapforce was an independent company founded by a former Google employee. It split in to two – Leapforce and Raterlabs few years back. Raterlabs was created exclusively to hire American search engine evaluators. Now Appen acquired Leapforce and hence both Leapforce and Raterlabs have now became part of Appen. That’s why you would see Appen logo and branding in the previous Leapforce website.

Also you will notice that Raterlabs website design and everything is same as current Appen/Leapforce account. 

Also note that the previous Leapforce website/company is also known as Appen Connect}

Who exactly will you be working for?

Although, you will be working for the aforementioned companies, the real client you will be working for is either Google or Microsoft.

In most cases it is Google.

Google contracts these companies mentioned above, and they in turn look for raters like you and me.

So, although you won’t actually be Google employee, you will indirectly be working for Google in making the world wide web a better place.

How to write CV for search engine evaluator position?

Now writing a good CV is pretty important to be considered for this position. Thousands of people apply to these companies every month. Standing out from them is pretty tough.

Here is my take on how to write a good CV.

  • Make it single page.
  • No need of cover letter.
  • No need to attach your photo.
  • Make sure to not leave any gap years  in your academic, professional details

How long does it take to hear back from them?

It can take anywhere from few days to few months. It all depends on their needs and demand from client.

People often complain that they are yet to hear back from them. Often times, the emails from Appen or Lionbridge ends up in spam folder. And when you get an email from them, the company name “Appen or Lionbridge are not immediately visible on mobile devices.

Where do we mostly check our emails these days? Mobile phones !

Take a note of these emails and make sure to check your spam as well once you apply to these companies

Here are the emails they could email you from –

Appen/Leapforce email addresses –


 Lionbridge email addresses –


Raterlabs – 

I don’t work here or don’t know of any one who works here. So you will have to check your email including spam and other folders regularly if you apply here. And look out for the keywords like qualification test, raterlabs, and so on.

Note that you won’t see the word Appen, Lionbridge or Raterlabs in the email preview unless you click and double check. So be a bit diligent in checking your emails after applying to these companies.

Where to work as one? 

Now this is a tough one to decide. Here is a comparison table –

Hires worldwide for all positions except Americans who wants to work as search engine evaluator.Hires worldwide for all positions except Americans who wants to work as search engine evaluator.Only for Americans
One can work for unlimited number of hours except for Brazilians who are limited to 10-20 hours per week.Their hours are limited, ranging from 20 – 30 hours per week.
They give bonus hours of up to 15 hours per week when the task load is high.
Fixed 26 hours per week
Apart from search engine evaluator position one can work in multiple projects at the same time.Has multiple projects but one person can only work at one project at a time.
Only search engine evaluator project is available.
Candidates are hired as independent contractors for 6 months. Candidates are hired as independent contractors for 12 months. Hired as part-time employees.
Pay rate ranges from $3/hr to $13 per hour.
Indians are paid $3/hr, Americans are paid up to $11/hr while Europeans are paid up to $13/hr
Pay rate ranges from $6.81/hr to $13 per hour.
Indians are paid $6.81/hr, Americans are paid up to $12/hr while Europeans are paid up to $13/hr

$11 - $13/ hr depending on which state you are.
Pays $25 per referral.No referral programThey do have a referral program
Paid via Payoneer under NET 15 days terms Paid via direct bank transfer under NET 45 days termsPaid via Payoneer/direct transfer under NET 15 days term

Do they hire in my country?

Well, they hire from almost any country in the world. You will have to check here whether they are hiring in your country or not.

They don’t hire from most African countries, Pakistan, Nepal, Bhutan, Sri Lanka, Bangladesh.

Do check it out here whether your country and language is in the list or not –

How much can you earn with search engine evaluator job?

You can earn anywhere from $500 – $3800 easily, depending on where you are located. Hey I made $2007 myself, being an Indian.

If you are an American working as a part-time employee who is limited to 26 hours per week, you can make up to

26*4*$11-$13 = $1144 – $1352

Brazilians used to be limited to 20 hours per week. But now since Appen brought in a lot of projects, there isn’t any weekly limit that they got to adhere to.

But everyone else can work as many hours as you want.

How much do they pay you?

As mentioned in the table above, payments ranges anywhere from $3 per hour to $13 per hour depending on where you are located.

Projects available inside?

All the companies got multiple projects inside except for Raterlabs.

Raterlabs is the only company that hires Americans for search engine evaluator position and it only has this one position open. Sure its website mentions openings for multiple countries with several projects. But trust me, its only for Americans and there is just one project available – Search Engine Evaluator aka Internet Analyst.

Appen has multiple projects apart from this. They offer Facebook projects (social media evaluator), Instagram projects and many others. Can Americans work in this projects? Yes, except that they won’t get the evaluator position to work for.

One can work for multiple projects at the same time in Appen.

Anyone who joins Appen after April 2019, and is selected to work as a search engine evaluator aka web search evaluator is eligible for $250 bonus. $150 for working at least 20 hours in the first two weeks and another $100 for working at least 20 hours in the next two weeks.

Lionbridge as multiple positions as well like Maps (Apple), Social media evaluator and so on. One cannot work for multiple projects at the same time.

One can also not work for different companies in the same project at the same time.

How to get paid?

If you want to work for Appen or Raterlabs, you need a Payoneer account. Here is a link to create one. You will get $25 bonus if you sign up via my link. You will get that bonus as soon as you are paid $1000 to your Payoneer account.


What about the qualification test?

There is a tough qualification test before you can work as search engine evaluator. It is one of the toughest exams that I have written in my life.

Hardly 1 out of 100 people gets through it i.e. only 1% qualifies for this position.

It is not a surprise considering the fact that the passing score is 87%

The qualification test consists of 3 parts –

  • Theoretical part
  • Practical part I
  • Practical Part II

How to pass the search engine evaluator qualification test?

Before appearing for the test, you will get a 164 page General Guidelines to study. It will be your Bible, before and after you pass the test.

As soon as you get it, read it like hell. You will need at least 20 hours to cover all of the concepts and go through all the examples.

Here is my recommended time distribution for the three part qualification test –

  • 1 and 1/2 day for the Theory part.
  • 2 and 1/2 day for Practical I
  • 3 and 1/2 day for Practical II

If you fail the test, you may get a one time retake.

Why people fail at search engine evaluator qualification test?

Most of the people fail at the third part of the exam. It is the toughest of all the parts. Some people fail even at the first part.

There are many reasons why people fail.

  • They don’t study enough or can’t study enough.
  • They don’t give much time or don’t have much time understanding the concepts.
  • They rush things through.
  • No attention to minute difference between different types of rating.

To help people crack this test, I build a course, two courses in fact –

One is for complete newbies who got no idea what search engine evaluation is.

The other one is for those who failed the qualification test . You can check it out here –


Thousands apply for this positions and getting a retake chance is getting harder and harder. So don’t take risk. Enroll in one of the courses above. Learn from someone who has been at this game for almost 3 years now. Learn the top 4 reasons why people fail and how to avoid those mistakes.

Final words

I have been working as Search Engine Evaluator for almost 3 years now.  It has benefited me tremendously. It has helped me in many ways –

  • I can figure out fake news very easily.
  • I now understand what is SEO and why certain pages rank high in Google search while others don’t.
  • I take every online info with a handful of salt.

According to me, this is the best work at home job for moms, students, introverts and for anyone who likes to work from home.

Remember, you will be working for Google indirectly. And you will be working to make the internet a better place for all.

So, go ahead, make your decision and sign  up in these three companies –

You also need a Payoneer account to get paid, so sign up here and get your $25 bonus*


Free Search Engine Evaluator Course

Do you want a free course on search engine evaluation? Do you want to learn about it from an active evaluator himself. Then click on the enroll button and get a premium lesson for free (limited time only)


After making almost $3000 online, I started this blog to document my own journey as a search engine evaluator aka Google rater, bug hunter, website usability tester, and blogger. Now I teach others how to make money online through genuine work from home opportunities. I've been featured in Huffington Post among others. You can read more about my personal struggles and successes in making money online here.

This Post Has 75 Comments

  1. gatorade

    Hello, Sam. Great stuff bro! Kudos.
    I had a question. For the social media evaluator post, does one require to have a real fb account?
    I never used social media as I was wary of giving out personal information and posting pics. I have a fake account though which I have been using for years. Can I use that ? Or do you need to link your fb account to them in some way? Last year I did do a social media related task where I had to note what sort of posts my friends had made in the recent past. If I remember correctly they didn’t ask me for a link to my profile. I got paid at the end of the task.

    1. gatorade

      Hello, Sam. I had another question. Appen has significantly reduced to 3$ their hourly rate. How much can one expect to make from the rater job in a month in Indian rupees on average. I ask because I read that you don’t have tasks available at all times.

      1. Sam Narzary

        Due to COVID-19, task flow had been very less in the month of April. This month is much better. One can easily average 100 – 160 hours in normal times.

    2. Sam Narzary

      In social media evaluator job, you would be rating from inside your FB profile. You don’t need to have your real name in the Facebook profile. It can be anything. And yes you would need to link your FB profile to their platform, to be more precise, they would need to add your profile to the rating tool.

      1. gatorade

        So, I can link my fake account and it won’t make the difference? i mean won’t it be an issue when they see the names don’t match? lol

        1. gatorade

          Oh and for the rater job, which one gets your vote? Appen or Lionbridge? You have been an old employee but the new ones will get a much reduced pay on Appen. So, taking that into account whom would you bat for?

        2. Sam Narzary

          Not a fake account. Your real Facebook account under a fake name works fine.

          1. gatorade

            Real fb account with a fake name? I don’t get it. Isn’t an account with a fake name on fb a fake? Can u elaborate? I have already read that article. I just wanted to know your personal opinion. haha
            Teemwork (earlier isoftone) pays 5$? They hire from India? How are they in comparison to Appen and Lionbridge? Appen pays much less but is there a better probability of getting tasks with them? Also, you say with Appen, one can get 100+ hrs of work. So, that means one can easily make 300$+?

          2. Sam Narzary

            Sign up with whatever Facebook account you have that you use regularly. You should be fine. Name on it doesn’t matter. My personal opinion is summed up in that article 🙂 Good Luck.

  2. Léo

    Olá Sam, tudo bem?
    Sou do Brasil e não falo inglês ainda, existe alguma forma de trabalhar para as empresas sem ter o inglês fluente? Existem empresas onde não seja necessário o inglês, e possamos utilizar mecanismos de tradução? Consiredo ter as demais habilidades necessárias para o trabalho.

    Desde já agradeço, até logo mais.

    1. Sam

      Hola. I don’t speak Spanish bro. Could you type in English please?

      1. Léo

        Hi Sam, how are you?
        I’m from Brazil and I still don’t speak English, is there any way to work for companies without being fluent in English? Are there companies in which English is not necessary and can use the translation mechanism? I consider having the maximum skills necessary for the job.

        Thank you in advance, see you later.

        1. Sam

          Si. Yes! You can work in your own language. Look these companies hire in the language of your country. In Japan, you can work in Japanese and in Brazil, you should be able to work in Portuguese. Just go ahead and apply. Good Luck Léo 🙂

          1. Léo

            Ok Sam, thank you guy.

          2. Sam

            Welcome 🙂

  3. Vipul

    Hi Sam,

    I had applied on Appen and received a mail from them asking me to digitally sign some documents on their site. After signing the document, I am taken to a page which shows a list of projects with a “Qualify” link against all the projects. What should I do here? Do I need to qualify for all these projects individually or do I need to select only one. And which one is to be selected if I select only one project?

    1. Sam

      You will need to select the project called “Yukon” if you want to work as a search engine evaluator. Yes, you need to hit “Qualify” button and qualify for each project individually.

  4. Brenz

    If you’ve worked for both Google and Microsoft companies through the hirers, were the tasks pretty much the same for Microsoft as with Google? I’m wondering is one better than the other overall or more interesting than the other?

    1. Sam

      I worked as an ad evaluator with iSoftstone,now known as I worked for like a month or two. I didn’t have time to handle both. Tasks were about Bing ads (Microsoft project). For Appen, I work as a search engine evaluator where we evaluate search results returned by Google. So we can’t really compare the two.

  5. Prakash

    Hi SAM,
    I am Prakash from Hyderabad India.

    I need help for the maintain the quality for long time. Please suggest me brother.

    I will wait for your valuable reply.

    1. Sam

      Read the guidelines, practice the simulations.. that’s it.

  6. Arun Chaurasia

    Hey Sam,

    Nice blog. I was working on a project in Appen till Dec 25, 2019, worked for it almost 4-5 months and one day suddenly they removed me from the project. I found that many people were removed without notice. I thought maybe it was because of quality, but later they sent an email telling me that it was not because of the quality, it was because the project ended. My question is are all projects are like this? Can they remove us anytime? Have you ever faced such situation there?

    1. Sam

      Not all projects are long term and not all projects are short term. Evaluator jobs like Yukon, Bane, Sanaga, Wouri are long term. Social media evaluator projects like Nile, Tahoe, and some one time projects are short term.

      Yes I was removed from Nile within 12 months. Happens. Just be sure to be in long term projects.

      1. Pallavi

        Hi Sam,

        Have you qualified Tahoe qualification quiz. I need your help. I am not able to qualify on it’s first attempt.
        If yes then do let me know at

        1. Sam

          No I haven’t. I think they give you two chances. So study hard and good luck. I heard that its test is not so tough.


    Give me the link of the social feed

    1. Sam

      It is inside Appen dashboard. Look for a tab called “Social” and you should be able to ask questions there.


    Hi SAM, I have recently qualified for a project in Appen. But I can’t start working as it is required SRT login to the workplace that I am unable to do as I don’t know what is my SRT email. Please tell me how to get it? I have sent a mail to the support team but not received any reply yet.

    1. Sam

      Could you post in the social feed of Appen please? I haven’t worked in a project that requires SRT log in years now. It used to be different long back. Post in the feed please, you should get correct guidance inside.

      1. Sam

        That sums up everything. And I am not Appen support who knows better right? They know it better. Wait for some time, I am sure they will send out SRT invite as soon as they need additional raters. I am deleting your above message as it contains sensitive info.

        1. PRASIT

          ok delete it

Leave a Reply