Edit: Have added some graphs at bottom of post.
Health warning - massively long answer here. There is a Conclusion/TLDR at the bottom, although I'd advise reading the key findings first, then the conclusion/TLDR.
I’m posting here with some information for people to digest and reflect on now that we’re a year out of Beta. I’ve chosen to post a new answer as opposed to amending my existing answer, as I’ve a lot to write here. For a lot of this answer and the metrics it provides, I’ve used Data Stack Exchange which can be used to query any Stack site and unearth metrics on the questions, users and all sorts of other things.
First, I ran the following query:
Select count(1)
FROM posts
WHERE posttypeid = 1
This shows that a total of 10,348 questions have been asked in the entire history of this site.
By amending this query slightly, I can determine the amount of questions that have been tagged as “identify-this”, and the amount of questions that have not been tagged “identify this”:
Select count(*)
FROM posts
WHERE posttypeid = 1 and tags LIKE '%identify%'
This shows the following metrics:
Total identify questions: 3,061 questions (30% of questions ever asked)
Total non-identify questions: 7,287 questions (70% of questions ever asked)
However, using the following query we can break these figures down a little more by studying the trend year by year:
Select count(*)
FROM posts
WHERE posttypeid = 1 AND creationDate betWEEN '20100101' and '20111231' and tags LIKE '%identify%'
In all the queries I’ve run, I’ve simply edited the creationDate field to change the date the question was asked from 2011, to 2012, to 2013, to 2014, to 2015 and finally to 2016. These are the results:
2011:
Total questions: 196
Total “identify” questions: 15 (8%)
Total “non-identify” questions: 181 (92%)
2012:
Total questions: 1421
Total “identify”: 188 (13%)
Total “non-identify” questions: 1233 (87%)
2013:
Total questions: 2015
Total “identify” questions: 471 (23%)
Total “non-identify” questions: 1544 (77%)
2014:
Total questions: 2585
Total “identify” questions: 888 (34%)
Total “non-identify” questions: 1697 (66%)
2015:
Total questions: 3591
Total “identify” questions: 1249 (35%)
Total “non-identify” questions: 2343 (65%)
2016:
Total questions: 497
Total “identify” questions: 242 (49%)
Total “non-identify” questions: 255 (51%)
Now, whilst 2016’s figures may look alarming, the reality is I haven’t broken down when different categories of questions are asked. In other words, it may well be the case that plot explanation questions are more popular later in the year, when summer blockbusters, or Oscar season, comes around. Regardless, the 2011 – 2015 figures show a noticeable trend, and if we take 2015, as the last complete year, as an indicator, identify-this questions comprise ~ 35% of total questions being asked on the site currently.
Now, at this point, I'm going to step away from the identify-this questions for a moment, and look at all the non-identify questions.
By editing the previous query slightly, we can reveal all the non-identify queries that have been asked in the history of the site that have been upvoted by users. In other words, all the non identify questions that have a vote score of 1 or more:
Select count(*)
FROM posts
WHERE posttypeid = 1 and tags NOT LIKE '%identify%' AND score > 0
Total non-identify questions: 7,287 questions
Total non-identify questions with a vote score of 1 or more: 6,737 (92%)
This is a great ratio. However, again I'm going to break it down by year, by amending the above query using the CreationDate field described previously. The results are as follows:
2011:
Total non-identify questions: 181
Total with upvote count > 0: 181 (100%)
2012:
Total non-identify questions: 1233
Total with upvote count > 0: 1219 (99%)
2013:
Total non-identify questions: 1544
Total with upvote count > 0: 1492 (97%)
2014:
Total non-identify questions: 1697
Total with upvote count > 0: 1570 (93%)
2015:
Total non-identify questions: 2343
Total with upvote count > 0: 2053 (88%)
2016:
Total non-identify questions: 255
Total with upvote count > 0: 190 (75%)
Again, we could arguably disregard 2016 given how early in the year it is, but we can clear see that in the “non-identify” questions, the upvotes on the questions are going down.
This could be due to lower user participation, it could be due to higher standards on the site. I’m simply indicating a trend here and people can read into it what they believe (or use more sophisticated queries than I’m capable of to produce more information on the subject).
All in all, it suggests to me that for the last few years, the trend has been that the total percent of “identify-this” questions has increased, whilst the total number of non-identify-this questions deemed worthy of upvoting by the community has decreased. Whether these two things are interlinked is not clear.
At this point, I'll return to considering identify this versus non-identify this questions and I'll look at the actual people posting these questions. The following query shows a join between the users table and the posts table:
Select count(*)
FROM posts inner join users on posts.owneruserid = users.id
WHERE posttypeid = 1
It returns a result of 10,204. The "users" table contains all the users active registered on the site. What this therefore means, is that out of the 10,348 questions ever asked by users, 10,204 of the questions have users who still have accounts on the site (99%).
I can then add a field to the query to determine how many users have more than 1 reputation point and how many have more than 124 reputation points:
Select count(*)
FROM posts inner join users on posts.owneruserid = users.id
WHERE posttypeid = 1
AND users.reputation > 1
By running the above query, and a slightly adjusted query with a different reputation score, I get the following results:
Total questions on the site: 10,348
Total questions asked with users remaining on the site: 10,204
Of the 10,204 questions asked by these users, # of askers with at least 1 reputation point: 9988 users (98%)
Of the 10,204 questions asked by these users, # of askers with at least 125 reputation points: 6854 users (67%)
I chose the score 124, because at 125 users can vote in elections, so I thought it might be a useful figure to determine community participation. Obviously it is a completely subjective figure and is not meant to indicate anyone with a lower score than that is not welcome in the community or is not part of the community. I am choosing to use it here to determine long-term participation in the site. Any user reading this with a score of < 125 - you are truly very welcome here.
By running a similar set of queries and specifically looking at “identify-this” and “non-identify this” questions, we can see the following:
Total identify questions ever asked: 3,061
Total identify questions ever asked by users remaining on the site: 3,042 (99%)
Of the 3,042 questions asked by these users, # of askers with at least 1 reputation point: 2887 (95%)
Of the 3,042 questions asked by these users, # of askers with at least 125 reputation points: 959 users (32%)
Total non-identify questions ever asked: 7,287
Total non-identify questions ever asked by users remaining on the site: 7162
Of the 7,162 questions asked by these users, # of askers with at least 1 reputation point: 7101 (99%)
Of the 7,162 questions asked by these users, # of askers with at least 125 reputation points: 5895 users (82%)
As you can see, participation in terms of reputation points is vastly inferior for askers of “identify this” questions compared to askers of “non-identify” questions.
More concerning to me, if we adjust the queries slightly and track this trend year-on-year, we get the following:
2011:
Total identify questions asked: 15
Total identify questions asked by users remaining on the site: 15 (100%)
Of the 15 questions asked by these users, # of askers with at least 1 reputation point: 15 (100%)
Of the 15 questions asked by these users, # of askers with at least 124 reputation points: 15 (100%)
Total non-identify questions asked: 181
Total non-identify questions asked by users remaining on the site: 180 (99%)
Of the 180 questions asked by these users, # of askers with at least 1 reputation point: 180 (99%)
Of the 180 questions asked by these users, # of askers with at least 124 reputation points: 180 (99%)
2012:
Total identify questions asked: 188
Total identify questions asked by users remaining on the site: 186 (99%)
Of the 186 questions asked by these users, # of askers with at least 1 reputation point: 186 (99%)
Of the 186 questions asked by these users, # of askers with at least 124 reputation points: 147 (79%)
Total non-identify questions asked: 1233
Total non-identify questions asked by users remaining on the site: 1218 (99%)
Of the 1,218 questions asked by these users, # of askers with at least 1 reputation point: 1218 (99%)
Of the 1,218 questions asked by these users, # of askers with at least 124 reputation points: 1170 (96%)
2013:
Total identify questions asked: 471
Total identify questions asked by users remaining on the site: 469 (~100%)
Of the 469 questions asked by these users, # of askers with at least 1 reputation point: 461 (98%)
Of the 469 questions asked by these users, # of askers with at least 124 reputation points: 197 (42%)
Total non-identify questions asked: 1544
Total non-identify questions asked by users remaining on the site: 1489 (96%)
Of the 1,489 questions asked by these users, # of askers with at least 1 reputation point: 1487 (~100%)
Of the 1,489 questions asked by these users, # of askers with at least 124 reputation points: 1325 (89%)
2014:
Total identify questions asked: 888
Total identify questions asked by users remaining on the site: 880 (99%)
Of the 880 questions asked by these users, # of askers with at least 1 reputation point: 833 (95%)
Of the 880 questions asked by these users, # of askers with at least 124 reputation points: 248 (28%)
Total non-identify questions asked: 1697
Total non-identify questions asked by users remaining on the site: 1674 (99%)
Of the 1,674 questions asked by these users, # of askers with at least 1 reputation point: 1668 (~100%)
Of the 1,674 questions asked by these users, # of askers with at least 124 reputation points: 1339 (80%)
2015:
Total identify questions asked: 1249
Total identify questions asked by users remaining on the site: 1243 (~100%)
Of the 1,243 questions asked by these users, # of askers with at least 1 reputation point: 1192 (96%)
Of the 1,243 questions asked by these users, # of askers with at least 124 reputation points: 312 (25%)
Total non-identify questions asked: 2343
Total non-identify questions asked by users remaining on the site: 2311 (99%)
Of the 2,311 questions asked by these users, # of askers with at least 1 reputation point: 2281 (99%)
Of the 2,311 questions asked by these users, # of askers with at least 124 reputation points: 1686 (73%)
2016:
Total identify questions asked: 242
Total identify questions asked by users remaining on the site: 242 (100%)
Of the 242 questions asked by these users, # of askers with at least 1 reputation point: 193 (80%)
Of the 242 questions asked by these users, # of askers with at least 124 reputation points: 40 (17%)
Total non-identify questions asked: 255
Total non-identify questions asked by users remaining on the site: 255 (100%)
Of the 255 questions asked by these users, # of askers with at least 1 reputation point: 232 (91%)
Of the 255 questions asked by these users, # of askers with at least 124 reputation points: 166 (65%)
Now, there is A LOT to take in in what I've written above. To summarise some key salient findings:
As the years have progressed, the askers of the majority of identify-this questions have kept their accounts active. However, they have increasingly become less involved in the site. 79% of the askers of identify-this questions in 2012 have 125 reputation points or more. Just 25% of askers of identity-this questions in 2015 have 125 reputation points or more.
Participation among "non-identify this" question askers appears to be contracting. 96% of the askers of non-identify this questions in 2012 have 125 reputation points or more. Just 73% of the askers of non-identify this questions in 2015 have 125 reputation points or more.
Now, I fully admit people from 2015 haven't had as long to accumulate 125 reputation points as those from 2012. However, 125 reputation points isn't that much, any even light users of the site can gain that amount through even semi-regular usage quite quickly.
Summary of key findings:
In my eyes, there is a few clear trends on the site:
Firstly, identify-this questions, whilst comprising 30% of overall questions ever asked, are becoming more numerous year on year in comparison to non-identify this questions.
Secondly, the quality of non-identify this questions is arguably dropping, based on the amount of upvotes given to them (although this is subjective and could simply be due to an increased volume of questions leading to people not having time to upvote as many questions). I believe it is the former, but acknowledge that this is subjective and others may argue differently.
Thirdly, the number of askers of identify-this questions who are active users of the site (with 125 reputation points or more) has decreased year on year, from 79% in 2012, to 25% in 2015.
Fourth, the number of askers of non-identify-this questions who are active users of the site (with 125 reputation points or more) has decreased year on year, from 96% in 2012 to 73% in 2015.
Conclusion / TLDR:
I love this site, even though I have been away from it for quite some time. However, I am concerned about the direction it is heading in. We don't appear to have a plan on how we want to drive the site forward, or the types of questions we want to see. More and more there appears to be a divide between "identify-this" and "other". I'm not sure what the solution to this is, or if anyone/everyone will agree with the many metrics I've provided here, but to me it suggests we need a site-wide decision on what sort of community we want to be, because it appears a large number of core users are unhappy about identify-this, but the metrics show the unhappiness people have towards these questions is not slowing the number of these questions being posted (rather it is increasing them).
- Do we simply ban them all?
- Do we ban all unless they provide
certain key information (e.g. a year, a country, a genre, an actor
description etc, even if people know the answers based without this
key information)?
- Do we allow them all and accept the site may well
become the web's premier haven for identify-this questions, leading
to either a spin off movie stack site or a much smaller
plot-explanation core on this site?
I don't know what the answer is, but I think we need to address it as the current approach doesn't appear to be working.
Even shorter TL:DR
Identify-this bad.
Other good.
beats chest
Edit:
I wanted to add some graphs for good measure. Here is the breakdown of identify this v other questions over the history of the site, with trendlines:
Here is a broken down version, showing JUST 2015 (and the first month of 2016):
As stated before, the trend isn't great, and banning them would be hard work, and would remove ~30% of the questions on the site (although I would still favour this approach).
