Sunday, 13 July 2014

Junethack 2014 is over!

June is finally over, and with it Junethack. Check out the individual scoreboard here and the clan scoreboard here.

I managed to take out the most unique deaths trophy (my clan consisted of just me), and also to ascend once. I had planned to ascend more often, but clan demilichens put up some pretty fierce competition for unique deaths, so I had to focus solely on it.

Woohoo!
clan demilichens put up very strong competition
Somehow I managed to get the second fastest ascension (by turns) for nethack4

A big thanks to all the organisers of Junethack, and also to member jonadab (from demilichens) who was the main unique deaths player from that clanned, and with whom it was very fun competing.

I definitely had a lot of fun going for unique deaths (after 500 or so they start to become quite hard to get, short of startscumming for wands of polymorph which I didn't do (it is perfectly acceptable, but I have a personal preference not to)).

I have a few comments to make, most (all) of which relate to unique deaths. You can see a list of all unique deaths attained throughout the month here.

I was personally proud of these deaths:

  • "killed by touching {artifact}", where I managed to get every aligned artifact (not just Quest artifacts). Looots of fountain quaffing. I was the only clan/person to obtain all of these.
  • "pissed off deity", obtained by #pray-ing in nethack1.3d until the deity smites you. One other clan did obtain this but I told them how to do it.
  • "disintegrated by amateur-hour horseshit": as an incantifier in dnethack, die of oversatiating while hallucinating. Incantifiers are a new race in dnethack who eat magic rather than food.

Some fun

Overall:

  • 12919 games were played, of which
  • 6416 were actually played, and
  • 6503 were scummed (quit or escaped on turn <= 10).

The top 3 most games played:

Most games played
Junethack.Username N Clan N
jonadab 1228 1 demilichens 2126
coffeebug 746 2 overcaffeinated 1312
coffeebeetle 566 3 BlackjackAndHookers 620
Wooble 462 4 TeamSplat 366
Ecthel 327 5 Justice 327

Who was responsible for the ~50% of scummed games? Note: these could be attempts at conducts or particular unique deaths, like rolling a wizard until you get a wand of polymorph to zap your pet with to get an out-of-depth monster. Not all players agree with start-scumming (for example in the case of unique deaths I didn't do any), but to be honest it is a perfectly valid/accepted tactic.

Most games scummed
Junethack.Username N Clan N
Ecthel 3101 1 demilichens 3146
DeuceofJune 2244 2 Goonsinjune 2247
the88kod 858 3 hi 52
nooodl 47 4 Justice 21
timco 37 5 Smile_Mold 16

What does the average nethack game look like?

variant Average length game (ascension) Average length game (death) Number of ascensions Number of games Rate (%)
vanilla 39704 669 108 8217 1.31
nethack4 39411 6168 20 116 17.24
acehack 36304 739 16 1345 1.19
unnethack 31184 1991 8 1125 0.71
dnethack 76620 2607 1 239 0.42
grunthack 50257 483 1 1418 0.07
sporkhack 40914 3006 1 169 0.59
oldhack NaN 368 0 290 0
  • dnetgames are very long
  • no-one ascended oldhack
  • nethack4 had the highest rate (it's not popular; people tend not to play it unless they specifically wanted the ascension for a trophy, for example)
  • vanilla is the most played variant (no surprises there)
  • only 1 ascension for dnethack, GruntHack and sporkhack (stth...)!
  • GruntHack and oldhack games are very short, probably because oldhack was mainly used for its unique deaths, and GruntHack was also a major unique-death variant.

Some fun death analysis

Note: see also Jonadab's unique death comments.

Here I'll just focus on the overcaffeinated (777) and demilichens (767) clans as there was a decent break between these two and the rest (but very well done to BlackjackAndHookers (341) too).

In total, 12919 unique deaths were obtained over the month.

It's hard to achieve a different death every time you play. Here's how we went. The "unique deaths (proposed)" column lists unique deaths under the scheme I propose later.

statistics
clan games played unique deaths (raw) unique deaths (junethack) unique deaths (proposed)
demilichens 5272 888 (17%) 767 (15%) 458 (9%)
overcaffeinated 1313 812 (62%) 777 (59%) 520 (40%)

Note: demilichens did not exclusively focus on unique deaths, so disregard their percentages.

Favoured variants

For jonadab (the main deaths-player from demilichens) and I, here are some further stats:

Breakdown of games played per variant

As you can see, we both used GruntHack heavily (it has the most verbose death messages), jonadab used acehack heavily whereas I used NAO heavily. I didn't play any vanilla games on the acehack server, and jonadab didn't play vanilla games on the NAO server.

Favoured classes

What classes do we like to deathscum in? (restricting to vanilla classes only):

Favourite roles for death scumming

I heavily favour arcs because of the ability to grab them and dig down in order to find harder monsters (E takes care of most monsters you /don't/ want to die to).

Wizards are popular for two reasons:

  • to use acehack's ability to reroll the starting inventory for a wand of polymorph. The wand can then be zapped at the pet to obtain out-of-depth monsters to die to.
  • they start with a cloak of magic resistance, so if you want to die to "M[rs]. Foo, the shopkeeper" as opposed to their wands (recorded in most variants as "killed by a (wand|magic missile)"), the cloak of magic resistance lets you survive the wands until the shopkeeper starts melee-ing you. (Shopkeepers all start with at least a wand of striking, and 25% chance for a wand of magic missile).

Priests can also be popular in order to die to the "wrath of {deity}" for deities that are unavailable as starting deities. For example, one can never start with a neutral knight: to become neutral, you must convert. Priests, however, get a random deity from the pantheon. Hence if you really wanted to be killed by Brigit you could play a knight and hope to find somewhere to convert relatively early on, or just keep rolling neutral priests until you got Brigit.

Deadliest deaths

As mentioned earlier, it's hard to be killed by something different each time. What were we killed by the most? Note: from now on, I will concentrate on just jonadab as the demilichens representative, because he was the main deaths-player and seemed to focus just on deaths. Including other demilichens members whose sole purpose was /not/ to obtain unique deaths would misrepresent the statistics.

deadliest killers (combined)
unique.death.junethack coffeebug jonadab
killed by a dwarf 8 38
killed by a water moccasin 34 3
killed by a jackal 13 12
killed by a sewer rat 14 10
killed by the wrath of Thoth 3 20
deadliest killers (individual)
unique.death.junethack coffeebug unique.death.junethack jonadab
killed by a water moccasin 34 killed by a dwarf 38
killed by a water demon 19 killed by the wrath of Thoth 20
killed by a sewer rat 14 killed by the wrath of Odin 15
killed by a small mimic 13 escaped 14
killed by a fox 13 killed by the wrath of Anhur 14

No surprise that I was disproportionately killed by water moccasins and demons. I obtained 26 deaths to touching artifacts (and one "due to inadvisable haste" which required wishing for an artifact in dnethack). All of these required wishing bar Sting and Orcrist, so there was a lot of fountain scumming.

Jonadab was killed by dwarves a lot --- I'm not sure what to make of that. Perhaps expeditions to Minetown for all the shopkeepers one could die to there? I imagine the wrath of Thoth (neutral wiz) could be a way of ending a game that was not going well (Jonadab's favoured class was wizard as mentioned earlier)?

I'm not sure why I had so many deaths to E-able monsters (besides the water foo); perhaps overconfidence? In the case of mimics, they can kill a level 1 player in one hit (3d4 damage), particularly if you encounter a second one while running away from the first. I'm inclined to believe overconfidence, though. I did manage 6 minotaur deaths (one in nh1.3d, the others in the other variants) which I'm quite proud of; they were all at the Castle with level 1 or 2 characters.

Most common deaths (after cleaning)

Here are the most common deaths, after the suggestions I made have been implemented.

deadliest killers (individual)
unique.death.me coffeebug unique.death.me jonadab
killed by a shopkeeper's wand 74 killed by the wrath of a deity 268
killed by the wrath of a deity 52 killed by a minion of a deity 58
poisoned by a rotted corpse 51 killed by a shopkeeper 57
killed by a shopkeeper 42 poisoned by a rotted corpse 49
killed by a shopkeeper's magic missile 40 killed by a shopkeeper's wand 48

You can see we would have saved a lot of time dying to shopkeepers (me) or #praying for minions/smites (jonadab), or eating lots of rotted corpses.

Ideas for next time (unique deaths)

In light of the month gone by, I have some suggestions re unique deaths to cut down on those deaths that are not "truly different".

In general I think the competition went quite well, but there are definitely some improvements to be made next year (mostly to do with GruntHack's very verbose way of reporting deaths).

I'm not sure how far to take condensing the deaths though. If you can argue that "poisoned by a rotten {monster} corpse" should be condensed to "poisoned by a rotten corpse" for all monsters, why can't "killed by a {monster}" be condensed to "killed by a monster"? But then there would be no fun to it.

Note: many of these suggestions are already implemented in NAO's "reduced deaths" page for each player. See DeathRobin's for an example.

GruntHack's "killed by a falling {foo}" death

Must be converted to "killed by a falling object", to be in line with all the other variants (besides "killed by a falling rock", which comes from dying to a rock trap).

Otherwise one could simply keep renaming their playerfruit and die over and over to it (by throwing it up in the air and having it fall onto them, which causes some small amount of damage). This would generate an infinite number of deaths that are currently counted as unique. (This is the very reason "killed by kicking {foo}" is converted currently).

For interest, in June 2014 the only deaths of this nature were "killed by a falling slime mold", "killed by a falling statue of a newt", and "killed by a falling dart". I know I was certainly aware of this death as was demilichens, and after testing it a couple of times we mutually agreed not to do it (it would explode the number of possible deaths given then large number of items easily available in nethack. Particularly with GruntHack's addition of materials to the item name, e.g. 'plastic pick-axe', 'gold pick-axe').

Shopkeeper deaths

There are many, many shopkeepers that one can die to. In all the variants but GruntHack, you need a way to survive the shopkeeper's wands in order to be killed by the shopkeeper themselves (e.g. be a wizard since you start with the CoMR).

With GruntHack's verbose death reporting (next section), the number of deaths that can be obtained just from shopkeepers explodes, and isn't really worth it.

I suggest to change "M[rs]. Foo, the shopkeeper" to "a shopkeeper" (this is what the NAO "reduced deaths" does).

GruntHack's "killed by a {foo}'s {bar}" deaths

GruntHack is very verbose in its death reasons. If you are killed by a wand of striking zapped by a gnome, the other variants record this as "killed by a wand". GruntHack records this as "killed by a gnome's wand".

In general, this is great (you know specifically what killed you). For unique deaths, this means that in practice one can run around attacking every shopkeeper one finds and die to their magic missile or wand of striking (or if a wizard, the shopkeeper themselves once the wands run out). Every shopkeeper is guaranteed to have a wand of striking, and has a 25% chance to have a wand of magic missile.

Given the huge number of shopkeepers (some of the variants have more), this makes shopkeeper-related deaths (and wand deaths) quite tedious. In practice, the deaths are the same.

The case of shopkeepers can be largely dealt with with the above fix (all shopkeepers become "a shopkeeper"), but in general, one could convert "killed by {foo}'s {bar}" to "killed by a monster's {bar}".

This is to stop tedious deaths where one throws/gives their attack wand to the first intelligent monster they see so that it will zap the wand at the player (and kill them). Granted, in practice this may not be particularly easy to do (I haven't tried), and you'd want to be a wizard to improve the chance of starting with an attack wand.

The regex would also be hard to implement, due to valid deaths with apostrophes in them, such as

  • "killed by a gas spore's explosion" would become "killed by a monster's explosion"
  • "quit while already on Charon's boat" --> "quit while already on monster's boat"
  • "killed by Durin's Bane" (a monster in unnethack) --> "killed by monster's Bane"

These are clearly nonsensical, but also there could be too many counterexamples to simply exclude them in a regex.

Deaths of the form "killed by {foo}, while {bar}"

Vanilla nethack only has ", while helpless" deaths but the patched version of vanilla on NAO is more detailed in the types of helplessness, and some of the variants add even more.

Some of these are non-trivial to obtain (e.g. "while being scared by rattling") and I think that one of each death should count.

For example, "{kill message}, while {bar}" could be "killed, while {bar}".

A single death such as "killed by a gnome, while frozen by a monster's gaze" would count as the single death "killed, while frozen by a monster's gaze" (it is up to you to be killed by a gnome while not frozen in order to get "killed by a gnome").

There is actually a huge range of conditions that one can die while experiencing.

  • getting stoned: delayed stoning by a footrice
  • fumbling: while fumbling (fumble boots/gloves or ice, I think)
  • sleeping: sleeping (e.g. by a trap or spell or wand or potion)
  • hiding from thunderstorm: during a lightning strike on the Plane of Air
  • stuck in a spider web: stuck in a spider web trap
  • reading a book: reading a spellbook
  • being frightened to death: ghost from a milky bottle, or haunted temple
  • frozen by a potion: paralysis potion
  • sleeping off a magical draught: sleeping from the vapour of a sleep potion (quaffing is "while sleeping").
  • gazing into a mirror: you are a floating eye and looked into a mirror, freezing yourself
  • ringing a bell: you rang a cursed bell and summoned a nymph, then there's a chance you simply take some time to ring the bell.
  • jumping around: while jumping (either magical or non-magical)
  • moving through the air: from Newton's third law
  • taking off clothes: while a nymph is stealing armour from you
  • paralyzed by a monster: as a result of it hitting you (e.g. guardian nagas have a paralysing bite attack)
  • digesting something: while digesting a monster, e.g. polymorphed into a purple worm. You have a better chance of this if you have slow digestion as digestion is slower then.
  • frozen by a monster's gaze: frozen because (e.g.) you hit the ubiquitous floating eye
  • frozen by a monster: frozen because you hit a gelatinous cube (cv "paralyzed by a monster", it hit you)
  • frozen by a trap: some chest traps will freeze you.
  • being scared by rattling: skeletons rattle their bones when you #chat to them. Scary.
  • being terrified of a demon: when you summon a demon after chaotic same-race sacrifice, you are momentarily terrified.
  • trying to turn the monsters: while using #turn
  • praying: while #praying.
  • disrobing: while taking off armour
  • dressing up: while putting on armour
  • dragging an iron ball: while punished
  • pretending to be a pile of gold: while polymorphed into a mimic, and pretending to be a pile of gold.
  • unconscious from rotten food: self-explanatory
  • fainted from lack of food: self-explanatory
  • vomiting: if you eat a rotten egg, for example.
  • being scared stiff: being scared by Magicbane
  • opening a container: It takes one turn to open a container. I think if you are paralyzed by a trap you get that instead. Or perhaps tins, which take some time to open.
  • gazing into a crystal ball: Using a crystal ball takes time.
  • helpless: anything not covered by the above (while engraving, for example).

In vanilla I'm not sure that you can actually obtain deaths that involve you being polymorphed (because when you die you either revert to human form meaning you don't die, or if you're wearing an amulet of unchanging you always get "killed while stuck in creature form"). However I think some of the other variants have fixed this. For example in GruntHack if you are polymorphed into a paper golem and are caught in a fire trap, you will get "burned away" (or something like that) if you wearing the amulet of unchanging instead of "killed while stuck in creature form".

Killed by a ghost of {playername}

Strip the player names. Sure, it'd be hard to actually exploit this (you'd have to organise with a few other players to all play at the same time and die on particular levels in an attempt to load each other's ghosts), but these deaths are all really the same.

I suppose you could keep those ghosts whose names are hard-coded into the source (they are the names of the nethack devs), but again, they're the same deaths.

Food poisoning

When you eat a tainted corpse and get "poisoned by a rotted {monster} corpse", perhaps this could be converted to "poisoned by a rotted corpse". I suppose you need to draw the line at condensing deaths somehow (for example, one could argue that condensing these deaths was the same as condensing "killed by a {monster}" to "killed by a monster" for all monsters). But I think there are still enough unique deaths available that condensing these will encourage players to go for different types of deaths.

Deities

Many, many deaths of the form "killed by the wrath of {deity}". These are obtained by starting a character with the appropriate alignment and simply #pray-ing until smitten by your god.

One could condense these to "killed by the wrath of a deity" (a la nethack 1.3d's "pissed off deity"), though one could also argue that there are only a relatively small number of deities (39) so this is not much of an issue.

To obtain deaths to deities that can not normally be accessed (for example, a knight can never start neutral), you can either convert (in my opinion this requires some skill and hence these deaths should somehow be noted), or you can simply roll a neutral priest, who will get a random neutral deity, and hope that you get Brigit. The latter doesn't require skill, just luck and the willingness to keep rerolling until you get the right deity.

Minions of deities

Sometimes when you #pray to a deity they will send a minion to smite you. Minions can be any A bar ki-rin and Archon (lawful deities), any E (neutral), or any & (demon) that is not a lord or prince, and is either nonaligned or chaotically-aligned (for example, erinyes are lawful).

There are many "killed by a {monster} of {deity}" deaths that are somewhat tedious to obtain, because really the only way to do it is to #pray repeatedly and hope you get a minion before you get smitten. You could also farm the high altar for minions but it seems like a bit too much work to get all the way to the high altar only to summon a single minion to die to.

I think these could either be converted to "killed by a minion of {deity}" or "killed by a minion of a deity" or "killed by a water elemental of {deity}" for each minion. NAO's "reduced deaths" version uses the latter, though I prefer the middle.

Priests and priestesses

At the very least, priests and priestesses should be the same, i.e. "killed by a priest(ess) of {deity}". One could conceivably condense this to "killed by a priest(ess) of a deity", which I favour, but again there is the question of condensing too much.

GruntHack's change of the kill prefix

In GruntHack, rather than being "killed by a freezing sphere", you are "frozen by a freezing sphere". The kill prefix is changed to match the damage type ("frozen by", "burned by", "shocked by", ...).

This means that you can obtain both "frozen by a freezing sphere" and "killed by a freezing sphere" depending on which variant to play, though those of these are really the same death.

One could strip off the kill prefix "(killed|frozen|burned|shocked|disintegrated) by" completely, though I haven't thought about this hard enough to decide yet whether this could merge deaths that are different. One example that comes to mind is being killed by a black dragon hitting you, as opposed to being disintegrated by one. In this case vanilla nethack has 'killed by a black dragon' and 'killed by a blast of disintegration', but perhaps there are other similar cases.

Ahh, grunt again. For example, water elementals have engulfing attacks. You can be "killed by a water elemental" (it melees you to death) or "suffocated by a water elemental" (it engulfs you), and these are fundamentally different. Silly grunt!

GruntHack's Racial monsters

GruntHack introduces racial monsters, for example there are ogre werejackals, elven werejackals, ettin werejackals, ... Soldiers, sergeants, captains, lieutenants, mummies, zombies, vampire foo, werefoo, nurses, guards, prisoners, watchmen, watch captains, shopkeepers, and priests/priestesses can be racial, leading to a huge number of deaths by the same monster of different races.

I probably wouldn't bother condensing these, since it does appear that races add to the difficulty of the monsters and hence affect their generation. For example, ettin werefoo have a much higher difficulty level than kobold werefoo and are hence only encountered much deeper in the dungeon.

The one exception is in the case of shopkeepers and priest(esse)s. Shopkeepers have been addressed previously (and their races are not recorded in the death message). I think races should be stripped from priests because rather than having difficulties and so on, they are only generated in temples so to hope to encounter a good spread of the clergy in various races is unlikely.

Appendix: Implementation

Here's some code similar to Junethack's normalize_death code, but it implements my suggestions. It's written in R:

conversions made
from to
killed by a {foo}, while {bar} killed, while {bar}
killed by a hallucinogen-distorted {foo} killed by a hallucinogen-distorted monster
killed by a invisible {foo} killed by a invisible monster
killed by a falling {not 'rock'} killed by a falling object
ghost of {name} ghost
poisoned by a rotted {monster} corpse poisoned by a rotted corpse
wrath of {deity} wrath of a deity
priest/priestess priest(ess)
M[rs]. {name}, the shopkeeper a shopkeeper
killed by touching {artifact} killed by touching an artifact
killed by a {minion} of {deity} killed by a minion of a deity

Notes:

  • naming a monster strips everything after the name, including ", while bar" (if present). Otherwise I could name (say) my dog "foo, while helpless" and be killed by it to get "killed by a little dog named foo, while helpless" in an attempt to get "killed, while helpless".
  • "invisible hallucinogen-distorted" as a combination counts as unique.
  • deaths with ", while" attempt to preserve the first word. For example "burned by asdf, while helpless" and "killed by asdf, while helpless" counts as two different deaths. I'm not sure if this could lead to some nonsensical deaths.
normalize.deaths.extra <- function (deaths) {
    # sic. the 'killey' is a typo on that site.
    deaths <- gsub('^killey by an ', 'killed by a ', deaths)

    # convert 'killed by a foo, while bar' to 'killed, while bar'
    #  (up to you to be killed by them while *not* bar)
    # NOTE: preserve first verb, is this nonsensical?
    # NOTE: ', while' occurs AFTER 'named', so do not die to a named animal
    #       or you will lose this.
    deaths <- gsub('^([^ ]+).*, while (.*)$', '\\1, while \\2', deaths)

    # 'hallucinogen-distorted'
    deaths <- gsub('hallucinogen-distorted .*', 'hallucinogen-distorted monster', deaths)

    # 'invisible'
    deaths <- gsub('by (the )?invisible ', 'by ', deaths)
    deaths <- gsub('by (an|a) invisible ', 'by a ', deaths)
    deaths <- gsub('by invisible ', 'by a ', deaths)
    deaths <- gsub('by a invisible (hallucinogen-distorted )?.*', 'by an invisible \\1monster', deaths)
    
    deaths <- gsub(' (her|his) ', ' eir ', deaths)
    deaths <- gsub(' (her|him)self ', ' eirself ', deaths)
    deaths <- gsub(' (her|him)self$', ' eirself', deaths)
    
    # note: named strips everything after it, so don't try to get ', while bar'
    #  conditions with a named monster.
    deaths <- gsub(' (called|named) .*', '', deaths)

    deaths <- gsub(' \\(with the Amulet\\)$', '', deaths)
    
    deaths <- gsub('choked on .*', 'choked on something', deaths)
    
    deaths <- gsub('killed by kicking .*', 'killed by kicking something', deaths)
   
    # killed by a falling {foo}
    deaths <- gsub("^killed by a falling (?!rock).*$", "killed by a falling object", deaths, perl=T)

    # strip ghost names
    deaths <- gsub(" ghost of .*", " ghost", deaths)

    # food poisoning
    deaths <- gsub("^poisoned by a rotted .* corpse", "poisoned by a rotted corpse", deaths)

    # deities
    deaths <- gsub(" wrath of .*", " wrath of a deity", deaths)

    # priests and priestesses
    deaths <- gsub("priest(ess)?", "priest(ess)", deaths)

    # shopkeepers
    deaths <- gsub("M[rs]\\. [A-Z].*, the shopkeeper", "a shopkeeper", deaths)

    # a {monster}'s {item}: tricky because of general regex looking for `'`
    # e.g. Durin's Bane, gas spore's explosion, quit while already on Charon's boat
    # while frozen by a monster's gaze
    # deaths <- gsub("^(\\w+) by (an?|the) .*'s (?!explosion.*)$", "\\1 by a monster's \\2", deaths, perl=T)
    # narrow it down to wands only?

    # ?? touching (an artifact)?
    deaths <- gsub("killed by touching .*", "killed by touching an artifact", deaths)
    
    # ?? minions of deities? "minion of a deity"? "minion of {deity}"? "{minion} of a deity"?
    # the first: 1 death. the second: 39 deaths (#gods). third: #minions deaths.
    deaths <- gsub("(\\w+ elemental|Aleax|couatl|Angel|\\w+ demon|\\w+ devil|(suc|in)cubus|balrog|pit fiend|nalfeshnee|hezrou|vrock|marilitherinyes) of [A-Z].*", "minion of a deity", deaths)

    # ?? racial priest*s?

    # return
    deaths
}

Appendix: filter unique deaths

The implementation for Junethack 2014, ported to R. Taken from normalize_death.rb on the junethack GitHub repository

normalize.deaths <- function(deaths) {
    # sic. the 'killey' is a typo on that site.
    deaths <- gsub('^killey by an ', 'killed by a ', deaths)
    deaths <- gsub(', while .*', '', deaths)

    deaths <- gsub('hallucinogen-distorted ', '', deaths)
    
    deaths <- gsub('by (the )?invisible ', 'by ', deaths)
    deaths <- gsub('by (an|a) invisible ', 'by a ', deaths)
    deaths <- gsub('by invisible ', 'by a ', deaths)
    
    deaths <- gsub(' (her|his) ', ' eir ', deaths)
    deaths <- gsub(' (her|him)self ', ' eirself ', deaths)
    deaths <- gsub(' (her|him)self$', ' eirself', deaths)
    
    deaths <- gsub(' (called|named) .*', '', deaths)
    
    deaths <- gsub(' \\(with the Amulet\\)$', '', deaths)
    
    deaths <- gsub('choked on .*', 'choked on something', deaths)
    
    deaths <- gsub('killed by kicking .*', 'killed by kicking something', deaths)
    
    # deaths <- gsub('^killed by (the|an?) ', 'killed by ', deaths)
    return(deaths)
}

Appendix: data

If you're after data for all games from participating players in Junethack, I am making it available in two formats:

  • all bundled as an R RData file as data.frames (or data.tables, if you care to load that package):

    load('junethack.2014.rda') # tables deaths, users, junethack.users, clans

  • an archive of CSVs. Comma-separated, string fields quoted (there should not be embedded quotes, but if there are they are escaped), non-values are 'NA' (no quotes).

The RData file is at https://bitbucket.org/mathematicalcoffee/junethack-analysis/downloads.
The CSV zipfile is at https://bitbucket.org/mathematicalcoffee/junethack-analysis/downloads.

There are 4 tables provided. Yes, there is redundancy between the tables because I know nothing about databases and found it useful in writing these posts.

The tables do not include games that Junethack counted as "junk games" (explore, polyinit, debug, or multiplayer games).

(The script used to generate these files is here; it worked for me but is not guaranteed to work for you, and is not really in a release-ready state. Use at your own risk).

deaths table

Sample from the deaths table (continued below)
mode exp xplevel align0 gender0 endtime starttime carried event turns conduct death name align
Cha Mal 2014-06-20 09:29:19 2014-06-20 09:24:38 1170 3976 killed by a rothe coffeebug Cha
Neu Mal 2014-06-04 01:22:42 2014-06-04 01:22:39 1 4095 quit DeuceofJune Neu
Table continues below
gender race role uid birthdate deathdate deaths maxhp hp maxlvl deathlev deathdnum points version server variant
Mal Gno Bar 5 2014-06-20 2014-06-20 1 38 -2 4 4 2 794 0.6.3 sporkhack.com sporkhack
Mal Gno Wiz 5 2014-06-04 2014-06-04 0 11 11 1 1 0 0 3.4.3 nethack.alt.org vanilla
Table continues below
realtime achieve flags dnetachieve endtimeus starttimeus temporary intrinsic extrinsic charname unique.death.junethack unique.death.me
280 0 killed by a rothe killed by a rothe
2 0 0 quit quit
startscummed clan
FALSE overcaffeinated
TRUE Goonsinjune

This is essentially the xlogfile for each server, filtered to contain just games by Junethack participants. See the nethackwiki pages for the Xlogfile and logfile for information on these fields. If the xlogfile for a particular server does not support that value, it is listed as NA.

Columns of note:
* `death`: the original death.
* `starttime`, `endtime`: start and end time of the game. In the CSV formats they are left as-is (seconds since UNIX epoch); in the Rdata file they are POSIXct objects with GMT timezone.
* `name`: the character's name, but on the nethack4 server it is the name of the nethack4 account and `charname` is the character's name (on nethack4 you can pick your character name freely of your account name).

Columns I've added:
* `server`: the server on which the game was played
* `variant`: name of the variant
* `clan`: the clan this user was from (if any)
* `startscummed`: whether the game is considered startscummed or not (reason for death was escaped/quit and turns less than or equal to 10).
* `unique.death.junethack`: the death as Junethack counted it.
* `unique.death.me`: the death with my proposed changes.

users table

Sample from the users table
Junethack.Username Account Server Variant Clan
MrSnide MrSnide acehack.de acehack Justice
schistosomatic schistosomatic nethack.alt.org vanilla Justice

Information about users on Junethack. Note: when you register for Junethack, you pick an account name for Junethack (say 'coffeebug'). Then, on each server, you may register an account (from the server) to your Junethack username.

For example, I registered 'coffeebug' on NAO and 'coffeebeet' on the dnethack server.

The columns of the users table:

  • Junethack.Username: your Junethack account name (e.g. 'coffeebug')
  • Account: the name of the account on a specific server (e.g. 'coffeebeet')
  • Server: the name of the server that account is on (e.g. 'dnethack.ilbelkyr.de')
  • Variant: the variant (e.g. 'dnethack'. Note: the acehack.de server has oldhack, vanilla, and acehack variants and you can register an account for each).
  • Clan: the clan this Junethack user is in (if any).

If you look up all the rows for a particular Junethack.Username, that's just like the table on that user's Junethack page that lists their accounts on each server.

To look up all deaths for a given account on a server (e.g. all of my acehack.de acehack deaths), I'd look for rows in deaths where name was 'coffeebug' (my Account for acehack.de), server was 'acehack.de', and variant was 'acehack'.

To look up all deaths for my coffeebug Junethack account, I'd take all rows corresponding to Junethack.Username='coffeebug' from the users table, and join to deaths where the users (Account, Server, Variant) matches the deaths (name, server, variant).

junethack.users table

Sample from the junethack.users table
Junethack.Username Accounts Games Trophies Played Scummed Clan
coffeebeetle 2 567 1 566 1 overcaffeinated
coffeebug 8 746 19 746 0 overcaffeinated

This is essentially the Junethack users table (https://junethack.de/users), with the number of start-scummed games added and the user's clan.

The 'Junethack.Username' column here matches up with the same column from the 'users' table.

clans table

Sample from the clans table
Username Role User trophies Last game played (UTC) Clan
coffeebeetle member 1 2014-06-19 06:38 overcaffeinated
coffeebug admin 19 2014-06-30 15:01 overcaffeinated

This is just the 'Clan members' table from each clan page, joined together.

The 'Username' column here corresponds to the 'Junethack.Username' column in users and clans.

P.S. turns out that although I pronounce "deity" in the way that suggests the correct spelling, I have been spelling it "diety" for years. Hahaha!.

Posted by at 19:48 1 comments
Labels: , ,

Thursday, 26 June 2014

ggpie: pie graphs in ggplot2

How does one make a pie chart in R using ggplot2? (If you're impatient: see the final code here).

I know, I know, pie charts are often not very good ways of displaying data. It is hard to visually compare the relative sizes of slices (particularly the smaller ones and if they are scattered with larger ones in between), and you get no sense of scale.

However, sometimes a good ol' pie chart can convey your point in a way that many of the general public will find easy to understand. For example, this is how I've spent my "work" hours this week so far:

(I manually inserted the line break into "marking exams" so it wouldn't get cut off - not sure how to automatically do this).

Terrible, I know. But the plot conveys quite clearly the point I want to make: I wasted away almost half (!) of my week so far playing nethack when I should have been doing my PhD, which is much more time than I have spent on my PhD itself!

(Aside: nethack is a most excellent ASCII game that can be obtained here, or from your repositories on most Linux systems. In my defence, during the month of June the Junethack tournament is run and clan overcaffeinated (me) is in a deadly battle with clan demilichens to win the "most unique deaths" trophy. And this is the last week of June).

How to do it

My data looks like this (I've been using hamster time tracker, though I keep forgetting to track things.):

How I've spent my PhD hours this week
activity time
Nethack 15.2
PhD 7.4
Marking exams 4
Meetings 2.3
Lunch 2
Writing this post 1.5

My first attempt at building a pie chart of this data follows the ggplot2 documentation for coord_polar and this excellent post on r-chart. There are also a number of relevant questions on StackOverflow.

First we construct a stacked bar chart, coloured by the activity type (fill=activity). Note that x=1 is a dummy variable purely so that ggplot has an x variable to plot by (we will remove the label later).

(Aside: if x is a factor (e.g. factor("dummy")) for some reason the bar does not get full width, and the resulting pie chart has a funny little hole in the middle).

p <- ggplot(df, aes(x=1, y=time, fill=activity)) +
        geom_bar(stat="identity") +
        ggtitle("How I've spent my PhD hours this week")
print(p)

Then we use coord_polar() to turn it into a pie chart:

p <- p + coord_polar(theta='y')
print(p)

(Aside: does anyone find it weird that although my x axis was 1 and my y axis was time, on the resultant graph the x axis is now 'time' and the y axis is now 1?)

Let's tweak this so it doesn't look so bad.

First, some pure aesthetics: let's outline each slice of the pie in black using color='black' in geom_bar(). This causes there to be an ugly black line and outline on each square of the legend, so we remove that.

# We have to start the plot again because `color='black'` goes into geom_bar
p <- ggplot(df, aes(x=1, y=time, fill=activity)) +
        ggtitle("How I've spent my PhD hours this week") +
        coord_polar(theta='y')
p <- p +
        # black border around pie slices
        geom_bar(stat="identity", color='black') +
        # remove black diagonal line from legend
        guides(fill=guide_legend(override.aes=list(colour=NA)))
print(p)

Now, remove the axes:

  • both the axis labels ("time", "1"),
  • the tick marks on the vertical axis,
  • the tick labels on the vertical axis (0.75, 1.00, 1.25).

Note - for some reason, although 1 was our x variable, you remove its tick label by setting axis.text.y...

p <- p +
    theme(axis.ticks=element_blank(),  # the axis ticks
          axis.title=element_blank(),  # the axis labels
          axis.text.y=element_blank()) # the 0.75, 1.00, 1.25 labels.
print(p)

Now, I should also remove the '0', '5', '10', '15' tick marks/labels, being the cumulative number of hours spent so far, since they're not meaningful.

However, I want to label each slice of the pie, and it is convenient to put my labels in place of the '0', '5', etc.

In terms of their position, they should be located at the midpoint of each pie slice. Think back to the stacked bar chart I produced at the start, and recall that the y axis (time) shows cumulative hours spent.

This means the y coordinate of the end of each slice is given by cumsum(df$time) (cumulative sum of time spent so far), and so the coordinate of the midpoint of each slice is given by:

y.breaks <- cumsum(df$time) - df$time/2
y.breaks
## [1]  7.60 18.90 24.60 27.75 29.90 31.65

In order to implement this in the pie chart, we use scale_y_continuous with the breaks argument being the coordinates we've just calculated, and the labels argument being the activity name.

p <- p +
    # prettiness: make the labels black
    theme(axis.text.x=element_text(color='black')) +
    scale_y_continuous(
        breaks=y.breaks,   # where to place the labels
        labels=df$activity # the labels
    )
print(p)

Altogether now:

p <- ggplot(df, aes(x=1, y=time, fill=activity)) +
        ggtitle("How I've spent my PhD hours this week") +
        # black border around pie slices
        geom_bar(stat="identity", color='black') +
        # remove black diagonal line from legend
        guides(fill=guide_legend(override.aes=list(colour=NA))) +
        # polar coordinates
        coord_polar(theta='y') +
        # label aesthetics
        theme(axis.ticks=element_blank(),  # the axis ticks
              axis.title=element_blank(),  # the axis labels
              axis.text.y=element_blank(), # the 0.75, 1.00, 1.25 labels
              axis.text.x=element_text(color='black')) +
        # pie slice labels
        scale_y_continuous(
            breaks=cumsum(df$time) - df$time/2,
            labels=df$activity
        )

TL;DR

I've tied it together into a function ggpie:

library(ggplot2)
# ggpie: draws a pie chart.
# give it:
# * `dat`: your dataframe
# * `by` {character}: the name of the fill column (factor)
# * `totals` {character}: the name of the column that tracks
#    the time spent per level of `by` (percentages work too).
# returns: a plot object.
ggpie <- function (dat, by, totals) {
    ggplot(dat, aes_string(x=factor(1), y=totals, fill=by)) +
        geom_bar(stat='identity', color='black') +
        guides(fill=guide_legend(override.aes=list(colour=NA))) + # removes black borders from legend
        coord_polar(theta='y') +
        theme(axis.ticks=element_blank(),
            axis.text.y=element_blank(),
            axis.text.x=element_text(colour='black'),
            axis.title=element_blank()) +
    scale_y_continuous(breaks=cumsum(dat[[totals]]) - dat[[totals]] / 2, labels=dat[[by]])    
}

For example:

library(grid) # for `unit`
ggpie(df, by='activity', totals='time') +
    ggtitle("A fun but wasteful week.") +
    theme(axis.ticks.margin=unit(0,"lines"),
          plot.margin=rep(unit(0, "lines"),4))

(Note: for some reason the pie plots have an unreasonably large amount of white space, and the theme(*.margin) settings are an attempt to control that. However, I still get a lot of vertical space that I'm not sure how to compress).

Clearly the labels could do with more work (if they are too long they go out of the plot boundary, or they bump into the pie chart), but not bad for an hour and a half's work :)

Posted by at 00:27 17 comments
Labels: , ,
Subscribe to: Posts (Atom)