I believe your posts are taking this thread (“Unofficial AISB Loebner Prize 2017 Finalist selection”) far afield and deserve to be put in a different thread if you would like to continue the discussion.

But to play along:

Your priors are wrong.
If all of the finalists are equally weighted (.25) then all non finalist would be 0. If so, you could add them all and not impact the probability of a win by a finalist.

This competition had a classical “Long tail” distribution in quality of competitors which would impact the probability.
https://en.wikipedia.org/wiki/Long_tail

Thanks for your opinion, Merlin.

OK, we know the probability has to equal to one.

Indeed, 4 times .25 does equal 1.

However, a non finalist did gain entry to the final round.

So, that makes it 5 times .20 which also equals 1

Let’s use marbles in place of contestants…

If we have a bag of four unique color marbles, and we add
a fifth marble of the same color as one of the unique
color marbles, then how does that affect the probability
of a same color marble being picked, compared to a
unique color marble being picked?

∞Pla•Net - Sep 23, 2017:

OK, we know the probability has to equal to one.
However, a non finalist did gain entry to the final round.

Bad assumption, but ok.

Let’s use marbles in place of contestants…

If we have a bag of four unique color marbles, and we add
a fifth marble of the same color as one of the unique
color marbles, then how does that affect the probability
of a same color marble being picked, compared to a
unique color marble being picked?

That is where the analogy runs amiss. The bag can only hold 4 marbles.

Prelim=
C1-Blue
C2-Red
C3-White
C4-Orange
...
CN-Yellow

Now since the bag only holds 4 marbles, you could take C1 and CN and replace the blue with a green. But that would leave:
Finals=
C1-Green
C2-Red
C3-White
C4-Orange

But that is not really what you are after.

The probabilities would be closer with:
C1 27 Blue
C2   23 Red
C3   21 White
C4   20 Orange

CN 12 Yellow

But you would also need to account for any prior overlap between Blue and yellow.

(For those interested:Bayes Theorem)

Carl B - Sep 19, 2017:

from the article - Sep 18, 2017:

...His three victories in the Loebner prize notwithstanding, Worswick is ready for a new challenge. “Only the arrogance of the human race believes nothing can be more intelligent than a person — and I’m sure back in Turing’s day this was the case, but it’s now time to move on.”

Care to elaborate Steve?

Certainly Carl. Let’s take the question, “What is the population of Brazil?”
Bot: The current population of Brazil is 209,637,523 as of Wednesday, September 20, 2017, based on the latest United Nations estimates. Brazil population is equivalent to 2.77% of the total world population. Brazil ranks number 5 in the list of countries (and dependencies) by population.
Human: No idea but quite a few million I would guess.

The bot’s answer is more intelligent but not humanlike. One of the dislikes I have about the Turing Test is having to dumb down Mitsuku to give “I don’t know” type answers instead of intelligent ones. I would much rather see a “best chatbot” contest than one based on deception and lies.

I echo Steve’s desire for a best chatbot contest.

I have a similar concern that for most contests, all chatbots need to have a copy of all human knowledge instead of a central repository that each bot could just query.

For Midge, I plugged in a dictionary (that was set a bit to high for the finals).

Ha… ha,ha! Merlin…

I suggested before in my hypothesis they should have been averaged. You’re analogy to blur Blue and Yellow to a color average, Green suggests C2-Red is actually C1-Red!  Clearly C1-Green did not exist in the Prelim. Therefore, it obviously was not in the finalist selection.  This supports a theory that C1-Green was what is known as a Ringer, since C1-Green substituted for C1-Blue in the competition.

Congratulations to C1-Red!

https://medium.com/@charlie.e.moloney/how-to-win-a-turing-test-the-loebner-prize-3ac2752250f1

A judge’s report on the contest. Of note are his description of how he crashed Rose, and another anecdote rather debunks Prof. Kevin Warwick’s paper what claimed that machines could pass the Turing Test if they simply kept quiet. At least Uberbot’s malfunction thus served some purpose, one could say.

If intelligence becomes key instead of ‘human-like’, wouldn’t the contest have to allow entries from Alexa, Google, Siri and dare we forget the colossus, Watson and all the other ‘intelligent’ entities out there? Wouldn’t Internet access become necessary? Doubt there’d be many winners at Bletchley.

I would like to see decent conversational exchanges tested over a 3 - 5 minute period using a variety of preselected topics like news, feelings, thoughts about a subject, dating or human interaction, philosophical issues, general dialog, etc. instead of knowing how high some building might be or what the temperature of the sun is at its core…things a lot of humans don’t know.

Otherwise, you might as well hook up all the cables to the Internet and let them slug it out with which could compile the most knowledge in the shortest time.

Just my thoughts.

The AISB is already considering opening up internet access:
http://www.access-ai.com/news/3964/turing-test-future-look-like
To address obvious concerns: It is possible to monitor or restrict internet access to allow only trusted domains. The Winograd Schema Challenge had such an arrangement. Many entries could benefit from access to Wikipedia, Wordnet, Framenet, and other well-established resources.

Art Gladstone - Sep 25, 2017:

If intelligence becomes key instead of ‘human-like’, wouldn’t the contest have to allow entries from Alexa, Google, Siri and dare we forget the colossus, Watson and all the other ‘intelligent’ entities out there? Wouldn’t Internet access become necessary? Doubt there’d be many winners at Bletchley.

Intelligence should be the ultimate goal, as being “human-like” could be seen as a deficiency (do you really want to hear an “oops” from a bot after asking it to do something important for you?).

“Information is the lowest form of intelligence”, and stand-alone bots are barely more than NLP to interpret input + a natural language generator using a db of static information to output a response.  Not being able to have access to the Internet IMHO makes the Loebner Prize contest archaic and un-compatible with modern cloud computing-dependent AI (used by the real players like Google, MS, Apple, Amazon, Laybia AI, etc.) that has the potential for high-level intelligence (supra-human even).

Although I realize great personal effort is put into the Loebner Prize contest, the contest should be either scrapped or radically revised since it now seems to have basically become a vehicle to promote a private company (Pandorabots). This is especially evident by the lack of news coverage or even general interest (outside of the small group of bot-masters) in the Loebner Prize contest.

Steve Worswick - Sep 23, 2017:

Carl B - Sep 19, 2017:

from the article - Sep 18, 2017:

...His three victories in the Loebner prize notwithstanding, Worswick is ready for a new challenge. “Only the arrogance of the human race believes nothing can be more intelligent than a person — and I’m sure back in Turing’s day this was the case, but it’s now time to move on.”

Care to elaborate Steve?

Certainly Carl. Let’s take the question, “What is the population of Brazil?”
Bot: The current population of Brazil is 209,637,523 as of Wednesday, September 20, 2017, based on the latest United Nations estimates. Brazil population is equivalent to 2.77% of the total world population. Brazil ranks number 5 in the list of countries (and dependencies) by population.
Human: No idea but quite a few million I would guess. The worst answer is a canned response derived from

The bot’s answer is more intelligent but not humanlike. One of the dislikes I have about the Turing Test is having to dumb down Mitsuku to give “I don’t know” type answers instead of intelligent ones. I would much rather see a “best chatbot” contest than one based on deception and lies.

I agree that the “Human-like” goal encourages “deception and lies” to win (which is unfortunately very human-like!) and the most “intelligent” bot would a better goal. Indeed, an intelligent bot could always “dumb down” a response to appear more human-like, with the option of asking for a detailed answer.  Using your example, a human-like reply would be something like the rounded actual number (The current population of Brazil is about 210 million humans), while a detailed reply would provide a summary of a Wikipedia entry (for example).

The only thing worse than an “I don’t know” response is a canned “cheater” response derived from NLP tree such as: What is the population of Brazil -> What is the population of x -> Answer: “The population of Brazil is (millions)”.  This leads to bogus answers to questions like: What is the population of my town-> What is the population of x -> Answer: “The population of your town is (millions)”.

You know “intelligence” has never been a goal of the Turing test. Is “thinking” intelligent? If you look on one side of the medal they give you for “winning” the contest, you will see Alan Turing and the words, “Can Machines Think?”

If a computer can’t be distinguished from a human being, could it be said to be thinking?

What is thinking?  The word “intelligent” appears only once in Turing’s 1950 paper: “Intelligent behaviour presumably consists in a departure from the completely disciplined behaviour involved in computation, but a rather slight one, which does not give rise to random behaviour, or to pointless repetitive loops. Another important result of preparing our machine for its part in the imitation game by a process of teaching and learning is that “human fallibility” is likely to be omitted in a rather natural way, i.e., without special “coaching.” (The reader should reconcile this with the point of view on pages 23 and 24.) Processes that are learnt do not produce a hundred per cent certainty of result; if they did they could not be unlearnt.”

The word “intelligence” also appears only once (other than the name of the paper), Turing proposed creating a child program that could learn from a teacher. “One may hope, however, that this process will be more expeditious than evolution. The survival of the fittest is a slow method for measuring advantages. The experimenter, by the exercise of intelligence, should he able to speed it up. Equally important is the fact that he is not restricted to random mutations. If he can trace a cause for some weakness he can probably think of the kind of mutation which will improve it. “

Here, Turing ascribes “intelligence” to the teacher, not the child.

Robby Garner - Sep 25, 2017:

You know “intelligence” has never been a goal of the Turing test. Is “thinking” intelligent? If you look on one side of the medal they give you for “winning” the contest, you will see Alan Turing and the words, “Can Machines Think?”

Good point about the goal of the “Turing Test” not being to get “Intelligence” from a computer.  However, Alan Turing probably did not foresee the advent of ubiquitous digital (and thus easily stored and retrieved) communication in global social networks that are being continuously harvested of data, generating unimaginable amounts of “thoughts” by actual humans. Since much of this info is stored in cloud computing clusters and can be used to train, eg, neural-network-based modules to generate area focused “thought” processors. My basic point is that the “Mimic Test” as it currently is defined by Turing may be a somewhat dated concept given the 21st-century computing landscape.

You could be right Carl, about the dated nature of the Imitation Game. Turing’s descriptions of the “child” are quaint in that he has no idea the kinds of editing and programming abilities that we now have.

I’ve enjoyed reading this thread. I agree with you and especially what you said about the deevolution of LPC to a vehicle for Pandorabots. I agree with Tom about some things too. I think Steve is sincere about what he creates, and is not swayed by the intentions of others while he is making his art.

Well, since 2010 the winner has always been either Bruce or Steve, with Mohan Embar being the one exception. I don’t expect that to change as they basically have a 10 year head start on newcoming developers, and the bigger players have more useful pursuits than to fail at being human. This makes for a rather stale contest.

I think all would be served with less focus on unintelligent human aspects, while retaining the challenge of natural conversation. I also think much will depend on whether the Loebner Prize can find itself a new sponsor.