Statistics on identification questions and their askers

Question

Identification questions are a major point of discussion on this site and we all have our own subjective impressions about them. Therefore I would like to have some more hard numbers and statistics about them. Those are not necessarily to be used as the sole hard determinant for making any conclusions about a definite policy on those questions, but just to have a little more information to base any possible discussions on.

This is partly inspired by this related question on Ainme & Manga Meta (a site which seems to share our fate to a large degree) and this similar but much older question on our meta.

So what are some interesting statistics about the development of identification questions and their primary users and those users' further contributions to this site?

Answer 1

Now first of all, what is an "identification question" or "ID question"? Quite simply, it is any question tagged with a tag of the form identify-this-..., in particular the tags identify-this-movie, identify-this-tv-show, identify-this-actor and identify-this-episode. For assessing those statistics, I'll mostly employ the site's real-time search features as well as queries to the Stack Exchange Data Explorer (SEDE). Note that this could result in slight discrepancies, since the SEDE databases are only updated once a week (a few hours before writing this answer) and don't track everything, like deleted questions or user registration. Also keep in mind that we have a policy for deleting inactive unanswered ID questions which might "prettify" the results a little bit in favour of ID questions.

_{I'll also try to add a few simple first conclusions in little quote boxes where appropriate. Those are however not to be mistaken for the ultimate truth behind those statistics but merely immediate and direct conclusions to further the understanding of the data a little.}

---

What about the overall situation of identification questions?

Let's first take a look at some overall numbers of how many of those we have at all. There are currently 2,490 undeleted ID questions, which is 29.1% of all the 8,570 undeleted questions. But what about deleted questions? Of all 3,004 deleted questions 54.8% are identification questions.

Next, what are some overall average stats of those questions (e.g. number of views, score, answers, ...) in comparison to all questions and all non-ID questions?

_{ID questions and their answers seem to be of much lower interest and of lower value to the community while answering behaviour seems to be more similar across all questions.}

So after looking at the current situation, what is the general development of identification questions? For this we can employ a simple SEDE query to look at the percentage of ID questions among all undeleted questions tracked over time (left: absolute amount of questions, yellow: all, blue: ID, red: non-ID, right: perecentage of ID questions):

We can also take a look at the percentage of ID questions asked per week or per month:

_{ID questions enjoy an overall quite steady increase in frequency and relative amount. And in the last month there were more ID questions asked than any other kind of question.}

So much to the deleted and undeleted questions. What about the closed questions. We have a specific close-reason for identification questions that lack in detail/effort (or are about music videos or commercials for that matter). For this we can employ the closing statistics available to 10k users, which only go back 90 days at maximum though. From the 739 ID questions asked in the last 90 days 205 (27.7%) were closed for not enough detail (8 of which got edited after closing and 4 reopened after improvement). Furthermore, this specific ID close-reason makes up for 52% of all closed questions in the last 90 days, which is however also roughly the ratio of ID questions received in that period.

_{We close quite some many ID questions for lacking in detail. However, we also close a similar ratio of all the other questions for a variety of reasons.}

What about the users primarily asking those questions?

Now let's take a look at the users. 74.8% of all ID questions were asked as a user's first question (72% as a user's first overall post, including any previous answers). Furthermore, from all users' first posts, 34% were either ID questions (27.5%) or answers to an ID question (6.5%). So those 34% of posters presumably came here for the topic of identification. However, we can't measure how many users were attracted by those questions but didn't post anything in this category first.

_{While ID questions are often asked by new users, the ratio of new users attracted by them does not seem much higher than the overall ratio of those questions.}

But what about user retention? How many of those users come back to the site? For this, let's take a look at when users have last visited the site compared to when they asked their first question. Specifically, what percentage of users still came back X days after they asked their first question:¹

So after 1 month 33.1% of the users who asked an ID question as their first question are still around, whereas from the users who started with a non-ID question 66.7% are still active.

_{Users starting with ID questions disappear more often/faster than users starting with other kinds of questions.}

But staying on the site is one thing, what about further contributions to the content? For this we again take a look at the users starting with an ID question vs. the users starting with a non-ID question and how many of those users have more posts than this question and how many of them have more that are not also ID questions. In addition to that we can also take a look at the average amount of questions, answers and further non-ID posts those two user groups have, as well as their average reputation:¹

_{Users starting with ID questions have a lower chance to contribute further to the site's content and have fewer contributions on average, including a lower average reputation.}

---

1) _{In this query I chose to leave out the first month of site lifetime due to a much less stable scope and site development and a lack of ID questions as frequent questions.}

Answer 2

I've already posted this as an answer here, but as it relates to this question, I'll repeat my answer:

Health warning - massively long answer here. There is a Conclusion/TLDR at the bottom, although I'd advise reading the key findings first, then the conclusion/TLDR.

I’m posting here with some information for people to digest and reflect on now that we’re a year out of Beta. I’ve chosen to post a new answer as opposed to amending my existing answer, as I’ve a lot to write here. For a lot of this answer and the metrics it provides, I’ve used Data Stack Exchange which can be used to query any Stack site and unearth metrics on the questions, users and all sorts of other things.

First, I ran the following query:

Select count(1)
FROM posts
WHERE posttypeid = 1

This shows that a total of 10,348 questions have been asked in the entire history of this site.

By amending this query slightly, I can determine the amount of questions that have been tagged as “identify-this”, and the amount of questions that have not been tagged “identify this”:

Select count(*)
FROM posts
WHERE posttypeid = 1 and tags LIKE '%identify%'

This shows the following metrics:

Total identify questions: 3,061 questions (30% of questions ever asked)
Total non-identify questions: 7,287 questions (70% of questions ever asked)

However, using the following query we can break these figures down a little more by studying the trend year by year:

Select count(*)
FROM posts
WHERE posttypeid = 1 AND creationDate betWEEN '20100101' and '20111231' and tags LIKE '%identify%'

In all the queries I’ve run, I’ve simply edited the creationDate field to change the date the question was asked from 2011, to 2012, to 2013, to 2014, to 2015 and finally to 2016. These are the results:

2011: Total questions: 196
Total “identify” questions: 15 (8%)
Total “non-identify” questions: 181 (92%)

2012: Total questions: 1421
Total “identify”: 188 (13%)
Total “non-identify” questions: 1233 (87%)

2013: Total questions: 2015
Total “identify” questions: 471 (23%)
Total “non-identify” questions: 1544 (77%)

2014: Total questions: 2585
Total “identify” questions: 888 (34%)
Total “non-identify” questions: 1697 (66%)

2015: Total questions: 3591
Total “identify” questions: 1249 (35%)
Total “non-identify” questions: 2343 (65%)

2016:
Total questions: 497
Total “identify” questions: 242 (49%)
Total “non-identify” questions: 255 (51%)

Now, whilst 2016’s figures may look alarming, the reality is I haven’t broken down when different categories of questions are asked. In other words, it may well be the case that plot explanation questions are more popular later in the year, when summer blockbusters, or Oscar season, comes around. Regardless, the 2011 – 2015 figures show a noticeable trend, and if we take 2015, as the last complete year, as an indicator, identify-this questions comprise ~ 35% of total questions being asked on the site currently.

Now, at this point, I'm going to step away from the identify-this questions for a moment, and look at all the non-identify questions.

By editing the previous query slightly, we can reveal all the non-identify queries that have been asked in the history of the site that have been upvoted by users. In other words, all the non identify questions that have a vote score of 1 or more:

Select count(*)
FROM posts
WHERE posttypeid = 1 and tags NOT LIKE '%identify%' AND score > 0

Total non-identify questions: 7,287 questions
Total non-identify questions with a vote score of 1 or more: 6,737 (92%)

This is a great ratio. However, again I'm going to break it down by year, by amending the above query using the CreationDate field described previously. The results are as follows:

2011:
Total non-identify questions: 181
Total with upvote count > 0: 181 (100%)

2012:
Total non-identify questions: 1233
Total with upvote count > 0: 1219 (99%)

2013:
Total non-identify questions: 1544
Total with upvote count > 0: 1492 (97%)

2014:
Total non-identify questions: 1697
Total with upvote count > 0: 1570 (93%)

2015:
Total non-identify questions: 2343
Total with upvote count > 0: 2053 (88%)

2016:
Total non-identify questions: 255
Total with upvote count > 0: 190 (75%)

Again, we could arguably disregard 2016 given how early in the year it is, but we can clear see that in the “non-identify” questions, the upvotes on the questions are going down.

This could be due to lower user participation, it could be due to higher standards on the site. I’m simply indicating a trend here and people can read into it what they believe (or use more sophisticated queries than I’m capable of to produce more information on the subject).

All in all, it suggests to me that for the last few years, the trend has been that the total percent of “identify-this” questions has increased, whilst the total number of non-identify-this questions deemed worthy of upvoting by the community has decreased. Whether these two things are interlinked is not clear.

At this point, I'll return to considering identify this versus non-identify this questions and I'll look at the actual people posting these questions. The following query shows a join between the users table and the posts table:

Select count(*)
FROM posts inner join users on posts.owneruserid = users.id
WHERE posttypeid = 1

It returns a result of 10,204. The "users" table contains all the users active registered on the site. What this therefore means, is that out of the 10,348 questions ever asked by users, 10,204 of the questions have users who still have accounts on the site (99%).

I can then add a field to the query to determine how many users have more than 1 reputation point and how many have more than 124 reputation points:

Select count(*)
FROM posts inner join users on posts.owneruserid = users.id
WHERE posttypeid = 1
AND users.reputation > 1

By running the above query, and a slightly adjusted query with a different reputation score, I get the following results:

Total questions on the site: 10,348
Total questions asked with users remaining on the site: 10,204
Of the 10,204 questions asked by these users, # of askers with at least 1 reputation point: 9988 users (98%)
Of the 10,204 questions asked by these users, # of askers with at least 125 reputation points: 6854 users (67%)

I chose the score 124, because at 125 users can vote in elections, so I thought it might be a useful figure to determine community participation. Obviously it is a completely subjective figure and is not meant to indicate anyone with a lower score than that is not welcome in the community or is not part of the community. I am choosing to use it here to determine long-term participation in the site. Any user reading this with a score of < 125 - you are truly very welcome here.

By running a similar set of queries and specifically looking at “identify-this” and “non-identify this” questions, we can see the following:

Total identify questions ever asked: 3,061
Total identify questions ever asked by users remaining on the site: 3,042 (99%)
Of the 3,042 questions asked by these users, # of askers with at least 1 reputation point: 2887 (95%)
Of the 3,042 questions asked by these users, # of askers with at least 125 reputation points: 959 users (32%)

Total non-identify questions ever asked: 7,287
Total non-identify questions ever asked by users remaining on the site: 7162
Of the 7,162 questions asked by these users, # of askers with at least 1 reputation point: 7101 (99%)
Of the 7,162 questions asked by these users, # of askers with at least 125 reputation points: 5895 users (82%)

As you can see, participation in terms of reputation points is vastly inferior for askers of “identify this” questions compared to askers of “non-identify” questions.

More concerning to me, if we adjust the queries slightly and track this trend year-on-year, we get the following:

2011:
Total identify questions asked: 15
Total identify questions asked by users remaining on the site: 15 (100%)
Of the 15 questions asked by these users, # of askers with at least 1 reputation point: 15 (100%)
Of the 15 questions asked by these users, # of askers with at least 124 reputation points: 15 (100%)

Total non-identify questions asked: 181
Total non-identify questions asked by users remaining on the site: 180 (99%)
Of the 180 questions asked by these users, # of askers with at least 1 reputation point: 180 (99%)
Of the 180 questions asked by these users, # of askers with at least 124 reputation points: 180 (99%)

2012:
Total identify questions asked: 188
Total identify questions asked by users remaining on the site: 186 (99%)
Of the 186 questions asked by these users, # of askers with at least 1 reputation point: 186 (99%)
Of the 186 questions asked by these users, # of askers with at least 124 reputation points: 147 (79%)

Total non-identify questions asked: 1233
Total non-identify questions asked by users remaining on the site: 1218 (99%)
Of the 1,218 questions asked by these users, # of askers with at least 1 reputation point: 1218 (99%)
Of the 1,218 questions asked by these users, # of askers with at least 124 reputation points: 1170 (96%)

2013:
Total identify questions asked: 471
Total identify questions asked by users remaining on the site: 469 (~100%)
Of the 469 questions asked by these users, # of askers with at least 1 reputation point: 461 (98%)
Of the 469 questions asked by these users, # of askers with at least 124 reputation points: 197 (42%)

Total non-identify questions asked: 1544
Total non-identify questions asked by users remaining on the site: 1489 (96%)
Of the 1,489 questions asked by these users, # of askers with at least 1 reputation point: 1487 (~100%)
Of the 1,489 questions asked by these users, # of askers with at least 124 reputation points: 1325 (89%)

2014:
Total identify questions asked: 888
Total identify questions asked by users remaining on the site: 880 (99%)
Of the 880 questions asked by these users, # of askers with at least 1 reputation point: 833 (95%)
Of the 880 questions asked by these users, # of askers with at least 124 reputation points: 248 (28%)

Total non-identify questions asked: 1697
Total non-identify questions asked by users remaining on the site: 1674 (99%)
Of the 1,674 questions asked by these users, # of askers with at least 1 reputation point: 1668 (~100%)
Of the 1,674 questions asked by these users, # of askers with at least 124 reputation points: 1339 (80%)

2015:
Total identify questions asked: 1249
Total identify questions asked by users remaining on the site: 1243 (~100%)
Of the 1,243 questions asked by these users, # of askers with at least 1 reputation point: 1192 (96%)
Of the 1,243 questions asked by these users, # of askers with at least 124 reputation points: 312 (25%)

Total non-identify questions asked: 2343
Total non-identify questions asked by users remaining on the site: 2311 (99%)
Of the 2,311 questions asked by these users, # of askers with at least 1 reputation point: 2281 (99%)
Of the 2,311 questions asked by these users, # of askers with at least 124 reputation points: 1686 (73%)

2016:
Total identify questions asked: 242
Total identify questions asked by users remaining on the site: 242 (100%)
Of the 242 questions asked by these users, # of askers with at least 1 reputation point: 193 (80%)
Of the 242 questions asked by these users, # of askers with at least 124 reputation points: 40 (17%)

Total non-identify questions asked: 255
Total non-identify questions asked by users remaining on the site: 255 (100%)
Of the 255 questions asked by these users, # of askers with at least 1 reputation point: 232 (91%)
Of the 255 questions asked by these users, # of askers with at least 124 reputation points: 166 (65%)

Now, there is A LOT to take in in what I've written above. To summarise some key salient findings:

1. As the years have progressed, the askers of the majority of identify-this questions have kept their accounts active. However, they have increasingly become less involved in the site. 79% of the askers of identify-this questions in 2012 have 125 reputation points or more. Just 25% of askers of identity-this questions in 2015 have 125 reputation points or more.

2. Participation among "non-identify this" question askers appears to be contracting. 96% of the askers of non-identify this questions in 2012 have 125 reputation points or more. Just 73% of the askers of non-identify this questions in 2015 have 125 reputation points or more.

Now, I fully admit people from 2015 haven't had as long to accumulate 125 reputation points as those from 2012. However, 125 reputation points isn't that much, any even light users of the site can gain that amount through even semi-regular usage quite quickly.

Summary of key findings:

In my eyes, there is a few clear trends on the site:

Firstly, identify-this questions, whilst comprising 30% of overall questions ever asked, are becoming more numerous year on year in comparison to non-identify this questions.

Secondly, the quality of non-identify this questions is arguably dropping, based on the amount of upvotes given to them (although this is subjective and could simply be due to an increased volume of questions leading to people not having time to upvote as many questions). I believe it is the former, but acknowledge that this is subjective and others may argue differently.

Thirdly, the number of askers of identify-this questions who are active users of the site (with 125 reputation points or more) has decreased year on year, from 79% in 2012, to 25% in 2015.

Fourth, the number of askers of non-identify-this questions who are active users of the site (with 125 reputation points or more) has decreased year on year, from 96% in 2012 to 73% in 2015.

Conclusion / TLDR:

I love this site, even though I have been away from it for quite some time. However, I am concerned about the direction it is heading in. We don't appear to have a plan on how we want to drive the site forward, or the types of questions we want to see. More and more there appears to be a divide between "identify-this" and "other". I'm not sure what the solution to this is, or if anyone/everyone will agree with the many metrics I've provided here, but to me it suggests we need a site-wide decision on what sort of community we want to be, because it appears a large number of core users are unhappy about identify-this, but the metrics show the unhappiness people have towards these questions is not slowing the number of these questions being posted (rather it is increasing them).

• Do we simply ban them all?
• Do we ban all unless they provide certain key information (e.g. a year, a country, a genre, an actor description etc, even if people know the answers based without this key information)?
• Do we allow them all and accept the site may well become the web's premier haven for identify-this questions, leading to either a spin off movie stack site or a much smaller plot-explanation core on this site?

I don't know what the answer is, but I think we need to address it as the current approach doesn't appear to be working.

Even shorter TL:DR

Identify-this bad.
Other good.
beats chest

Answer 3

I used the following query on the SEDE:

WITH
  first_user_posts AS (
    SELECT
      u.id AS userId,
      MIN(p.id) AS postId
    FROM users u 
    JOIN posts p ON (u.id = p.ownerUserId)
    WHERE p.posttypeid IN (1, 2)
    GROUP BY u.id
  ),
  first_ident AS (
    SELECT p.*
    FROM first_user_posts fup
    JOIN posts p ON (p.id = fup.postId)
    JOIN posttags pt ON (p.id = pt.postid)
    JOIN tags t ON (pt.tagid = t.id)
    WHERE p.posttypeid = 1 -- questions
      AND t.tagname LIKE 'identify%' -- identification requests
  )
SELECT
  COUNT(DISTINCT fi.owneruserid) AS nrs,
  CASE
    WHEN op.id IS NOT NULL THEN 1
    ELSE 0
  END AS has_other_post
FROM first_ident fi
LEFT JOIN posts op
  ON (   fi.id <> op.id
     AND fi.id <> op.parentid
     AND fi.owneruserid = op.owneruserid
     AND op.posttypeid IN (1,2))
GROUP BY
  CASE
    WHEN op.id IS NOT NULL THEN 1
    ELSE 0
  END;

First I select all first user posts per user, restricted to answers and questions. From those, I select the question that have a tag beginning with "identify".
Then I look for other posts made by the same user, excluding (of course) the ID request itself and follow-ups (self-answers) to the ID request, restricted again to questions and answers.

Currently the numbers are:

• 2217 users who posted just a single ID request
• 196 users who posted at least one other question or answer (which may have been another ID request)

Do these numbers make any sense?