ArticlePDF Available

The "Face with Tears of Joy" Emoji. A Socio-Semiotic and Multimodal Insight into a Japan-America Mash-Up

Authors:

Abstract and Figures

The blog site of the Oxford Dictionaries features a post dated November 16 2015, which announces that, “for the first time ever”, their “Word of the Year” is not a word, but a pictograph: the “Face with Tears of Joy” emoji. The term emoji, which is a loanword from Japanese, identifies “a small digital image or icon used to express an idea or emotion in electronic communication” (OED 2015). The sign was chosen since it is the item that “best reflected the ethos, mood, and preoccupations of 2015”. Indeed, the Oxford Dictionaries’ President, Caspar Grathwohl declared that emojis are “an increasingly rich form of communication that transcends linguistic borders” and reflects the “playfulness and intimacy” of global digital culture. Adopting a socio-semiotic multimodal approach, the present paper aims at decoding the many semantic and semiotic layers of the 2015 “Word of the Year”, with a special focus on the context of cultures out of which it originates. More in detail, the author will focus on the concept of translation as “transduction”, that is the movement of meaning across sign systems (Kress 1997), in order to map the history of this ‘pictographic word’ from language to language, from culture to culture, from niche discursive communities to the global scenario. Indeed, the author maintains that this ‘pictographic word’ is to be seen as a marker of the mashing up of Japanese and American cultures in the discursive practices of geek communities, now gone mainstream thanks to the spreading of digital discourse.
Content may be subject to copyright.
11
Hermes – Journal of Language and Communication in Business no 55-2016
Ilaria Moschini*
The “Face with Tears of Joy” Emoji. A Socio-Semiotic and Multimodal
Insight into a Japan-America Mash-Up
Abstract
The blog site of the Oxford Dictionaries features a post dated November 16 2015, which announces that, “for the rst
time ever”, their “Word of the Year” is not a word, but a pictograph: the “Face with Tears of Joy” emoji. The term
emoji, which is a loanword from Japanese, identi es “a small digital image or icon used to express an idea or emotion
in electronic communication” (OED 2015).
The sign was chosen since it is the item that “best re ected the ethos, mood, and preoccupations of 2015”.
Indeed, the Oxford Dictionaries’ President, Caspar Grathwohl declared that emojis are “an increasingly rich form of
communication that transcends linguistic borders” and re ects the “playfulness and intimacy” of global digital culture.
Adopting a socio-semiotic multimodal approach, the present paper aims at decoding the many semantic and semiotic
layers of the 2015 “Word of the Year”, with a special focus on the context of cultures out of which it originates. More in
detail, the author will focus on the concept of translation as “transduction”, that is the movement of meaning across sign
systems (Kress 1997), in order to map the history of this ‘pictographic word’ from language to language, from culture to
culture, from niche discursive communities to the global scenario. Indeed, the author maintains that this ‘pictographic
word’ is to be seen as a marker of the mashing up of Japanese and American cultures in the discursive practices of geek
communities, now gone mainstream thanks to the spreading of digital discourse.
Keywords
Geek Culture – Emojis – Emoticons – Kaomoji – Manga – Multimodality – Social Semiotics – Transduction
“I don’t consider [my drawings] pictures –
I think of them as a kind of hieroglyphics …
I’m writing a story with a unique type of symbol”
Osamu Tezuka
1. Introduction
The blog site of the Oxford Dictionaries features a post dated November 16 2015 which an-
nounces that, “for the rst time ever”, their “Word of the Year” is not a word, but a pictograph:
the “Face with Tears of Joy” emoji (see Figure 1).
Figure 1. The Face with Tears of Joy Emoji
* Ilaria Moschini
Department of Languages, Literatures and Intercultural Studies
University of Florence
Via S. Reparata, 93-95 – 50129 Firenze
Italy
ilaria.moschini@uni .it
Hermes-55-moschini.indd 11 26-07-2016 13:29:24
12
The term “emoji” is a loanword from Japanese (the plural of which can be both emoji or emo-
jis), a portmanteau that blends “e ‘picture’ and moji ‘letter, character” and identi es “a small dig-
ital image or icon used to express an idea or emotion in electronic communication” (OED 2015).
The word was chosen since it is the item that “best re ected the ethos, mood, and preoccupa-
tions of 2015”. Indeed, the Oxford Dictionaries’ President, Caspar Grathwohl declared that emo-
jis are “an increasingly rich form of communication, that transcends linguistic borders” and re-
ects the “playfulness and intimacy” of global digital culture, which is “visually driven, emotion-
ally expressive, and obsessively immediate” (Steinmetz 2015).
According to data from the Oxford Dictionaries Corpus and Oxford Dictionaries’ digital partner,
the mobile technology company SwiftKey (2015), the “Face with Tears of Joy” was the most used
emoji in 2015 (“20% of all the emojis used in the UK in 2015, and 17% of those in the US”), while
the word emoji itself has faced a similar increase, since “its usage more than tripled in 2015
over the previous year”. In addition to that, in March 2015, Instagram declared that digital lan-
guage has “evolved such that nearly half of comments and captions on [its platform] contain emo-
ji characters” (Davis/Edberg 2015).
Emojis thus appear to be a widespread method of communication that can potentially cross lan-
guage barriers and, even, enrich written language since, as the linguist Ben Zimmer af rms, emo-
jis can “function as a new form of punctuation” (Robb 2014).
Moreover, the acquisition by the Library of Congress (Allen 2013) of the intersemiotic ‘emo-
ji ed’ translation of Herman Melville’s classic, Moby Dick (Benenson 2010) may also suggest
that emojis can be used, like pictograms, to vehicle ideas not only emotions. This conceptual/
narrative usage of emoji characters has been underlined also by the electoral tweet sent by Sen-
ator Hillary Clinton in August 2015, where she exhorted her followers to express “in 3 emojis or
less” their feelings about student loan debts (see Figure 2).
Figure 2. Hillary Clinton’s Electoral Tweet
The present paper aims at decoding the many semantic and semiotic layers of the “Face with
Tears of Joy”, with a special critical focus on the cultural contexts out of which it originates.
More in detail, it will focus on the concept of translation as multimodal “transduction”, that is
the movement of meaning across sign systems (Kress 1997) in order to socio-semiotically “map
the history of the resource” of emojis (van Leeuwen 2004: 3) and its ‘translation’ from language
to language, from culture to culture, from niche discursive communities to the global digital sce-
nario. It is a theoretical standpoint that can be contextualized in the emerging interest in Transla-
tion Studies towards multimodality, especially with reference to texts that present a multi-coded
structure.
As regards its structure, the paper will start from an outline of the theoretical framework adopt-
ed; it will then concentrate on the rst set of emojis, those “pictographic words” created by the
Japanese telecommunication planner Shigetaka Kurita and on their main linguistic/semiotic and
cultural in uences.
Hermes-55-moschini.indd 12 26-07-2016 13:29:54
13
The text will proceed with the analysis of the globalization process of emojis, from their en-
coding in the Unicode system to their visual hybridization with the “smiley” symbol. Finally, it
will focus on the transpaci c intertextual chain of semiosis that pragmatically and historically
links emojis to emoticons and kaomoji, that are common markers of facial expressions created
with keyboard characters by niche sub-cultural communities, such as early computer users and
fandoms.
2. Intersemiotic Translation and Multimodal Studies
The English term “translation” derives from the Old French verb “translater” and/or directly from
the Latin “translatus”, the past principle of the verb “transferre”, that means “to carry over, to
bring over” (OED 2015). The etymology of the term reveals the foundational idea around which
the activity of translating is conceptualized in Western Countries (Tymoczko 2005a), that is to
“carry something across”, this ‘something’ being the meaning which is transferred across cultur-
al, national and linguistic borders.
The effect of such movements from a code to another code, from a country to another country,
from a culture to another culture usually implies a wide and varied spectrum of changes and al-
terations in the semantic and semiotic sphere. This is especially true in the contemporary scenario
that features the spreading of digitally mediated texts, which are constantly re-codi ed and re-sig-
ni ed giving rise to linguistic, semiotic and cultural hybrids.
As a consequence of the massive technological developments and of the related social process
of globalization, interests in Translation Studies have been increasingly moving towards the third
category of translation theorized by Roman Jakobson, namely “intersemiotic translation” (Ty-
moczko, 2005b: 1090). Indeed, in his seminal paper “On linguistic aspects of translation” (Jakob-
son 2004 [1959]: 139), the Russian-American structuralist identi ed three types of translation: in
the context of the same linguistic code, from language to language and from a signifying system
to another.
More in detail, Jakobson’s taxonomy featured the rst category as an “intralingual traslation”,
that is “an interpretation of verbal signs by means of other signs of the same language” (e.g. a pro-
cess of rewording a specialized jargon into common language). His second category, the “interlin-
gual translation” is a “proper translation” and refers to the activity of interpreting “verbal signs by
means of some other language”. His nal category, “intersemiotic translation”, or “transmutation”
was de ned as the “interpretation of verbal signs by means of signs of non-verbal sign systems”.
A de nition that draws on semiotics, that is the general science of communication through signs
and sign systems, of which language is but one (Munday 2012).
The above-mentioned link between Translation Studies and semiotics was favored by Jakob-
son’s use of Peircean semiotics, which is based on the assumption that all signs can be translated
into other signs (Stecconi 2010). Such initial connection can probably explain why Peircean se-
miotics gures prominently in research on intersemiotic translation (Gorlé e 1994, Torop 2000),
while other approaches like, for instance, those emerging from the functional and socio-semiotic
tradition such as Multimodality do not appear.
However, the situation has changed in the last decade since Translation scholars have dedicated
their attention to texts that feature a multisemiotic composition (e.g. Taylor 2004, Gambier 2006,
Snell-Hornby 2006, Dejardins 2008, O’Sullivan/Jeffcote 2013, Borodo 2015) and have advocated
for theories and frameworks that would help in the practice of translating artifacts, the design of
which features the structural combination of different semiotic modes (Kress/van Leeuwen 2001).
Understanding how people make meaning through various semiotic resources and the rela-
tionship of these practices with self-representation or with the representation of others is a major
focus of Multimodal Studies, especially in its social semiotic formulation (Jewitt 2009). Indeed,
a socio-semiotic approach to multimodality postulates a common communication trajectory that
involves three interconnected steps: what people want to signify, the relationship between what
Hermes-55-moschini.indd 13 26-07-2016 13:29:54
14
people want to signify and which ways of communication or signs are more suitable in given so-
cio-cultural contexts (Kress/Jewitt 2003).
The production of signs, that is a “combination of meaning and form” (Kress 1997: 6), is “mo-
tivated” (Hodge/Kress 1993) and people’s moves from mode to mode and from media to media
are driven by interests and by “what is possible to express and represent, readily, easily with a
mode, given its materiality and given the cultural social history of the mode” (Kress/Jewitt 2003:
14). “Mode”, which is a key term in Multimodal Studies, refers to the set of resources that people
in a given culture can use to communicate, while the word “media” refers to the channel of com-
munication (Kress 1997: 7).
This socio-semiotic and multimodal approach to intersemiotic translation will be used in the
present analysis, as previously said, to historically outline the chain of the subsequent resemioti-
zations (Kress 2000, Iedema 2001) of the “Face with Tears of Joy” starting from the creation of
the rst set of emojis.
3. The cultural and technological background of Japanese emoji
The Japanese telecommunication planner Shigetaka Kurita is credited with inventing emojis in
1999, when he was involved in the launch of a mobile Internet platform called “i-mode” (Blag-
don 2013).
At the time, Kurita was working at NTT Docomo, one of the largest mobile services in Japan
as a member of the “i-mode” team, the aim of which was to provide Internet service on a type of
mobile phones, the so-called “featured phones”, which had small LCD screens that could only
‘contain’ up to 48 letters. This monochrome site of display would have highly limited the ex-
change of information, and this is the reason why Kurita started to plan the creation of alphanu-
meric images that would take the emoticons (that had already gained popularity on the Internet)
to a strictly iconic level.
Kurita stated that the American AT&T was already offering information for cell phone users
but “everything was shown by text, even the weather forecast”, even if “Japanese weather fore-
casts have always included pictures or symbols to describe the weather” (Nakano 2015). Kurita’s
familiarity with this way of producing information led him to envision the possibility to create
text messages using images rather than words.
The rst set of emojis designed by him was made of 180 items that comprised not only smiley
faces representing emotions, but also objects and symbols such as light bulbs or bombs (see Fig-
ure 3). “I was working with the sense of creating a new alphabet – he declared – It was an attempt
to create texts rather than a sense of making pictures” (Nakano 2015).
Figure 3. Some of the rst emojis designed by Shigetaka Kurita, © NTT Docomo
For inspiration, Kurita turned primarily to “manga, kanjii characters and street signs” (Negishi
2014). As regards manga, that are Japanese comics, the inspiration Kurita derived from them was
Hermes-55-moschini.indd 14 26-07-2016 13:29:55
15
related primarily to their unique use of symbolic representation of emotions. Indeed, as Wall-
ested (2013: 5) explains:
Manga have a large diversity of metaphorical gure symbols called keiyu, that are not considered as
words or representational pictures, but act as symbolic adjectives or adverbs to events depicted. Keiyu
consist of manga symbols (manpu) and effect symbols (kouka). [These] symbols are applied to char-
acters or subjects as representational indicators denoting their physical (butsuriteki) states and/or as
metaphorical indicators connoting their psychological (shinriteki) states (Natsume/Takekuma, 1995).
Many of the metaphorical gure symbols mentioned in the quotation require an effort to be de-
coded, since their conceptual underpinnings are deeply rooted in Japanese cultural and visual
conventions, like the “gigantic sweat drop conveying embarrassment or nervousness”, that is
shown in Figure 4 (Cohn 2010: 7).
Figure 4. Visual Morpheme “Giant Sweat Drop”, source Morphology of Japanese Visual Language- re-
trieved at http://www.visuallanguagelab.com/A/jvlmorphology.html
As Wallestad explains, “one of the most used keiyu-manpu is the drop (suiteki)”, which can rep-
resent “both physical and psychological states” (2013: 6). Indeed, such a symbol can be applied
to different parts of the face, with its placement in uencing the meaning, and be associated with
“sweat, tears, saliva, nasal discharge, or water. [When] applied to the eyes, the drop(s) (suiteki)
become tears (namida) that can express sadness or overwhelming joy” (Wallestad 2013: 6).
The use of keiyu-manpu is a cultural and aesthetic tradition that, according to the French histo-
rian Jean-Marie Bouissou (2011 [2010]: 127), can be traced back to ancient woodblock printing
techniques, where movements were suggested by the eyes and the mouths of the represented sub-
jects and to the tradition of the Kabuki theatre, where a wide range of emotions was rendered by
the grotesque facial distortions of the actors.
The “Face with Tears of Joy” emoji appears to be consistent with such a visual vocabulary, es-
pecially with the symbolic representation of drops as namida, that are here used to express a
profound and unrestrained state of contentment.
As regards the in uence of kanji, the logographic Chinese characters adopted in Japan, Kurita
declared that he “took from kanji the ability to express abstract ideas” in a single character (Blag-
don 2013). As a matter of fact, ideograms derive from pictographs - that are pictorial representa-
tions for words and phrases - which imply a symbolic conceptualization of ideas. For what con-
cerns their shape, they all feature a squared form and are usually made up of two elements: the
hen”, which de nes the semantic area of the kanji, functioning thus as a sort of key to the gen-
eral meaning of the character, and a ‘body’ that speci es its actual meaning (Taylor/Taylor 1995).
Japanese traditional writing system seems to have deeply in uenced the structure of the ba-
sic unit of manga, where each element is not a neutral embellishment but, rather, a means of ex-
pression (Bouissou 2011 [2010]: 119). Indeed, as Hirofumi Katsuno and Christine Yano explain
(quoting Natsume/Takekuma, 1995), “a single frame of manga consists of not only simply draw-
ings and words, but a complex visual grammar of subject, object, word balloon, movement, back-
ground keiyu ( gure symbol) and on’yu (sound symbol)” (2002: 213).
Hermes-55-moschini.indd 15 26-07-2016 13:29:55
16
Together with the conceptual representation of objects and ideas, Kurita’s original emojis ap-
pear to have inherited from kanji characters the vertical orientation and their squared form.
Moreover, like their predecessors kaomoji (that are Japanese emoticons), emojis have drawn upon
the visual language of manga where the eyes are considered “the locus of facial expressivity”
(Katsuno/Yano 2002: 214).
The standardization of manga visual language is commonly attributed to Osamu Tezuka (Cohn
2010: 4), who is de ned as the “godfather of modern manga” and who had been greatly in u-
enced by the drawings of Walt Disney animated movies (Schodt 1983, 1996). As a matter of fact,
the salience of the eyes, which has become a distinctive feature of manga visual style, is an el-
ement that Tezuka derived from Disney and, more precisely, from the movie Bambi, where huge
round eyes were used to emphasize the emotive expressions of the faces (Pellitteri 2008: 198).
This element of manga visual design is a marker of the different and subsequent economic and
cross-cultural encounters between America and Japan that have occurred since the Nineteenth
century (Schodt 1994).
Going back to the creation of emojis, the modal affordance Kurita could use to design them
allowed him to ‘play’ only with a 12-by-12-pixel grid to translate images into a set of characters
able to be easily displayed on mobile screens. The result was a collection of very simple signs:
the original smiling face, for instance, featured up-side down Vs to indicate the eyes and a rec-
tangle that served as the mouth (see the rst emoji on the left side of the rst column in Figure 3).
Apart from the simplicity of the design due to the above mentioned technological restraints, if
we compare Kurita’s “happy face” with the “Face with Tears of Joy” emoji, we can easily see that
the form is different, since the “Face with Tears of Joy” is a yellow rounded sign.
In order to explain such a difference and to decode the socio-cultural and pragmatic meanings
associated to it, it is necessary to trace out other steps in the transduction across languages and
across cultures of the face sign that has become a pervasive component of contemporary global
digital culture, starting from its inclusion in the Unicode encoding system.
4. Going global: from Japan to the Unicode
In 1999 emojis began to be available on Japanese mobile phones and became immediately pop-
ular, although they created problems related to the fact that their codi cation had not been stand-
ardized. Indeed, since each mobile phone carrier had developed different (even if partially over-
lapping) sets of characters, each mobile vendor used their own text encoding extensions.
Figure 5. Mark Davis’ Keynote Speech at the 38th Unicode Conference (a snapshot), ©Unicode – retrieved
at https://www.youtube.com/watch?v=-n9ONNeACyw
As Unicode President and co-founder Mark Davis (2014) explained in his keynote speech at the
38th International and Unicode Conference, which was dedicated to “the past, the present and the
future of emojis”, there were codes that were associated to different images, but there were also
almost identical images that were identi ed by different codes, such as the penguins in Figure 5.
The result was that characters from different platforms could not always be displayed and emojis
Hermes-55-moschini.indd 16 26-07-2016 13:29:55
17
were replaced by the symbol of the question mark that was meant to signify the ‘untranslatability’
of the message. The situation worsened when non-Japanese mobile companies began to support
exchange with the original carriers (Davis/Edberg 2015).
It was Google that submitted an of cial request to the Unicode Consortium to have emojis
standardized since, in 2006, it had decided to enter the Japanese mobile market (Nasser/ Benen-
son et al., 2013). The interest in Japanese mobile market was due to the fact that it was more de-
veloped if compared to its American counterpart (Baron 2008: 128). However, it was only with
the release of Apple’s mobile operating system iOS 5 in late 2011, that emojis “made their real
international debut” (Blangdon 2013).
Technically, nowadays the term “emoji” refers to a single set of Unicode characters that is as-
sociated to an image (Davis/Edberg 2015). Unicode is an international computer industry stand-
ard for encoding and displaying most of the world’s writing systems that allows one operating
system to recognize text from another (Unicode Consortium 2015b). The Unicode represents the
foundation for software internationalization and standardization and, as Joseph D. Becker –
one of the co-founders – af rmed (1998 [1988]: 1), it was envisioned as “a new, worldwide AS-
CII” that is the American Standard Code for Information Interchange. Indeed, prior to the crea-
tion of Unicode, much of the computer world relied on the character encoding for English text,
which was the 7-bit ASCII. Since the people of the world, “need to be able to communicate and
to compute in their own native language, other than English”, with the spreading of computers
there was the rising of the necessity of an “international/ multilingual text encoding standard that
[was] reliable as ASCII, but that cover[ed] all the scripts of the world” (Becker 1998 [1988]: 2).
At semiotic level, the creation and the standardization of emojis deal with the transduction
of the message (e.g. the “Face with Tears of Joy”) across two different coding systems: the inter-
nal code de ned as hexadecimal (e.g. 1F602) and the glyph that is its graphic representation
(Unicode Consortium 2015a). While the rst has been fully standardized by the Unicode encod-
ing system, which means that for each conceptual image a unique identi cation number is fea-
tured, the graphical form of such a code varies with the implementation of emojis across different
platforms. Indeed, as one can see in Figure 6, the representation of the Unicode character 1F602,
named the “Face with Tears of Joy” features a range of visual interpretations.
Figure 6. Unicode Emoji Chart, (a selection) ©Unicode http://unicode.org/emoji/charts/full-emoji-list.html
The version of the 1F602 glyph chosen by the Oxford Dictionaries to indicate the ‘word of the
year” is the one designed by Apple, widely recognized as a de-facto standard (Nasser/Benenson
et al. 2013). It is a visual representation that, as we shall see in the next paragraphs, intertextually
links emojis to emoticons and to the famous symbol of the “smiley” face.
5. The “Smiley” and its optimistic commercial ethos
The history of the “smiley” face is “a narrative that encompasses big business, popular culture
and anti-establishment sentiment […] and a series of bitter copyright issues” (Piercy 2013). How-
ever, it is largely accepted that the original version of the sign (see Figure 7) was created in 1963
Hermes-55-moschini.indd 17 26-07-2016 13:29:55
18
by Harvey Ross Ball, an American freelance designer who was commissioned by the Worcester
Mutual Insurance Company to create a new logo in order to “raise morale among the employees
of an insurance company after a series of dif cult mergers and acquisitions” (Stamp 2013).
The image was reproduced on button badges for their employees to wear and to distribute to
clients to illustrate the new “service with a smile” company ethos (Piercy 2013). The structur-
al connection between the original “smiley” face and pin-back buttons as sites of display was
meant to reinforce the illocutionary force of the message, since badge pins – a US patent dated
1896 – are natively advertising media that were originally used for political slogans.
Figure 7. The original smiley face as designed by Harvey Ball © The World Smiley Foundation
Political buttons have been used in the United States since 1789, when George Washington’s sup-
porters wore buttons imprinted with a slogan during the rst presidential inauguration, while the
rst campaign buttons were used in 1824 (Badgeomania 2014). However, it is only at the begin-
ning of the Twentieth Century, that US companies started to exploit the communicative potential
of these patented pin-buttons, which functioned as billboards displaying persuasive messages in
the form of slogans or pictures.
Going back to the “smiley” button, it gained popularity at the beginning of the 1970s when,
combined with the saying “Have a Happy Day”, achieved its iconic status and became the “ubiq-
uitous icon of post-Vietnam America” (Piercy 2013). Through a chain of subsequent resemioti-
zations (Kress 2000, Iedema 2001) the image of the smiley face has featured a range of material
realizations from pin-buttons to t-shirts, from pillows and posters to ecstasy pills according to the
variation of the affordance of modes, of their materiality and of their grounding social, cultural
and aesthetic practices. Contextually, its meaning has changed, varying from the original optimis-
tic message of the Worcester Insurance Company, to become a commercialized logo, to an ironic
fashion statement, to a symbol of rave culture. However, the “smiley” appears to primarily em-
body the spirit of American capitalism and, more precisely, the optimistic commercial ethos we
mentioned above (Stamp 2013). And it is with this connotation that we nd it bloodstained and
critiqued on the cover of the rst number of the cult comic book series Watchmen, published by
DC Comics (Moore et al. 1986-1987).
Hermes-55-moschini.indd 18 26-07-2016 13:29:55
19
Figure 8. Cover of the rst number of Alan Moore’s Watchmen, 1986 © DC Comics
As regards its physical appearance, the “smiley” features a rounded form, which was probably
functional to its original material realization as a pin button and that recalls the conceptual sym-
bolic representation of a human face. The absence of perspective and the frontal angle ‘demand’
the viewers to enter into a friendly relationship (Kress/van Leeuwen 2006 [1996]: 145). The
friendship of the invitation is visually marked by the expression of the abstract face, where the
smiling mouth (especially in Ball’s original design) represents the most salient element in terms
of proportions. Another marker of happiness is the color, a bright fully saturated yellow that re-
inforces the ‘positive message carried by the mouth. Indeed, as regards the psychological and
cultural meaning of this color, van Leeuwen – quoting Faber Birren (1961: 143) – af rms “‘mo-
dern Americans’ see yellow as ‘cheerful, inspiring, vital, celestial’ and relate it to ‘high spirit’
and ‘health’” (2011: 56).
Such a semiotic con guration creates a visual form of direct address that asks the viewer to en-
ter into a relation of social af nity, thus implicitly demanding to share the same attitude and, prob-
ably, also to adhere to the same set of values. Indeed, it is a form of visual reciprocity which is
functional to the strengthening of ‘mutual’ relationship and af nity between the viewers and the
represented subjects that Kress and van Leeuwen (2006 [1996]: 118), along with Belting (1990:
57), date back to the devotional use of holy images in thirteen-century monasteries.
As regards Apple’s design of the emoji smiling faces (see Figure 6), their rounded shape and
their color recall Ball’s smiley, and also the shades (that can be clearly seen in Figure 1) suggest a
corporeity that reminds of the pin-button, as if the most relevant meaning were materially xed
in the original pin badge. At the same time, salience appears to be given to both the mouth and the
eyes, thus embodying, in this latest digital realization of the smiley, the mashing-up of two differ-
ent and culturally connoted loci of facial expressivity. It is a combination that can be potentially
related to the ‘convergence’ of discursive communities belonging to different cultural areas, such
as US computer scientist user groups, Japanese otaku and shōjo subcultures, American Sci-Fi and
manga/anime fandoms in the realm of global digital language, in the wake of subsequent trans-
paci c exchanges. It is an intertextual chain of semiosis that – as outlined in the next paragraph
– starts with emoticons and ends up in the creation of emojis.
6. Emoticons, Kaomoji and Emojis
Despite the resemblance of the word “emoji” to the English term “emoticon” (composed by the
words “emotion” and “icon”), which – according to the Oxford Dictionaries Blog has favored its
Hermes-55-moschini.indd 19 26-07-2016 13:29:55
20
memorability – emojis are different from emoticons since the latter indicate facial expressions
that are created using keyboard characters, while emojis are – as we have seen – pictorial signs.
In addition, the cultural origin of the two items of digitally mediated communication is different.
As a matter of fact, while emoticons stemmed out of the community of US computer scientist
user groups, the creation of kaomoji – that are Japanese emoticons and the ‘ancestors’ of Kurita’s
emojis can be traced back to youth culture, in particular to teenage female subcultures (shōjo)
and to obsessive fans of manga, otaku (Azuma 2010 [2001]).
The rst use of an emoticon in a computer message was reported by professor Scott E. Fahl-
man, who was – as he af rms – the rst to use a colon followed by a hyphen and a parenthesis to
represent a smiley face in 1982 (Krohn 2004). He posted the graphic representation of a facial ex-
pression in a message to an online electronic bulletin board during a discussion “about the limits
of online humor” and about the necessity “to denote comments meant to be taken lightly” (Lov-
ering 2007). In his message – the transcription of which can be read below1 – he pointed out the
necessity to give computer users a way to mark “jokes”.
19-Sep-82 11:44 Scott E Fahlman :-)
From: Scott E Fahlman
I propose that the following character sequence for joke markers: :-)
Read it sideways. Actually, it is probably more economical to mark
things that are NOT jokes, given current trends. For this, use :-(
Emoticons, thus, were created with the explicit function of being “humor indicators”, which are
fundamental – as the New Hackers Dictionary af rms – in on-line forums where the “lack of ver-
bal and visual cues can cause what were intended to be humorous, sarcastic, ironic, or otherwise
non-100%-serious comments to be badly misinterpreted” (Raymond 1993).
The reference to humor is also culturally connoted since humor and, more precisely, irony –
which involves playing with form – is a de ning feature of hacker subculture that “love[s] pranks,
clever programming tricks, toys and games” (Isaacson 2014: 201). Indeed, hackers celebrate val-
ues such as creativity, fun, informality, personal engagement, liberty, gaming and code mastery.
Another structural feature of computer culture – which is one of the four pillars of geek culture
(Konzack 2014: 59) – is the love of science ction and fantasy and the partaking in the related
fandoms, as Stewart Brand described in his Rolling Stone article dated 1972, “one of the rst and
still one of the most quoted descriptions of the Bay area computer scene” (Turner 2006: 116).
It seems that emoticons themselves can be traced back to the discursive practices of sci-
and fantasy fandoms; indeed, according to the astrophysics professor and science ction author,
Gregory Benford (1996), “most of the Net’s ‘emoticons’ […] had appeared in fanzines by the
1950s”. Such fandoms were mostly male communities (Jenkins 2013 [1992]) who used typed-on-
paper emoticons like Fahlman’s digital ones as visual clues of ironic adherence or disagreement.
In Japan, before the advent of commercial Internet that occurred in 1993, emoticons were fa-
miliar to the community of academic users who were connected to American and European uni-
versities and research institutes via a non-commercial network, JUNET (Japanese Unix Network).
On the other hand, general public users, who were connected through commercial networks, start-
ed to elaborate their own form of expressing emotions using keyboards and drawing on pre-exist-
ing narrative forms. As a matter of fact, many of otakus designed face marks (literally, kaomoji)
feature elements of manga visual style (Wallestad 2013: 8). More in details, they were devel-
oped in the sub-community of “early and heavy users of personal computers” that were typically
“young males with narrowly focused interests, such as computers or comics” and that were re-
ferred to as otaku, the colloquial for “geek” (Katsuno/Yano 2002: 201). In this initial stage, ka-
omoji were inserted at the end of the name, as if they were part of the signature. With the advent
of the commercial use of the Internet in Japan, kaomoji started to spread beyond the boundaries
1 For the complete retrieval of the entire discussion thread see http://www.cs.cmu.edu/~sef/Orig-Smiley.htm (De-
cember 2015).
Hermes-55-moschini.indd 20 26-07-2016 13:29:56
21
of otaku subculture. However, as Katsuno and Yano highlight (2002: 211), the true populariza-
tion of kaomoji “rested in the hands of the most popular pagers in Japan and female high-school
students”. Some scholars (Sugimoto/Levin 2000) have connected the Japanese female youth use
of kaomoji in text messages on pagers and, later on, on mobiles to the emergence of new literary
practices and to the rise of “kitten writing” in which women use kawaii (cute) culture (Kinsella
1995) to create new vernacular forms of language (Hjorth 2009: 68).
The contrast between American and Japanese face marks is mainly expressed by the direction
of the symbol, since the emoticons are read at perpendicular angles with the words – i.e. : ) –,
while kaomoji are read in line with words, like (^_^); and by the main locus of expressivity, that in
America is the mouth (hence, smiley), while in Japan is the eyes (Katsuno/ Yano 2002: 206). We
may also argue that, as far as the pragmatic use of such symbols is concerned, originally the two
elements of computer-mediated communication (CMC) features different purposes: as a matter of
fact, the main aim of Fahlman’s emoticons was to provide the meta-communicative frame ac-
cording to which messages have to be decoded; while kaomoji appear to natively present a visual,
conceptual and diegetic function.
Indeed, studies in CMC suggest that – also in contemporary digital discourse – emoticons are
used primarily not as markers of emotion (Baron 2000, Crystal 2001, Walther/D’Addario 2004),
but as indicators of the pragmatic illocutionary force of the utterances they are co-textually re-
lated to (Markman/ Ōshima 2007; Dresner/Herring 2010). It is the same function envisioned by
Fahlman when he proposed to use emoticons as ‘irony markers’.
On the other hand, kaomoji – being rooted in manga visual language and in kanji writing –
appear to be more performative and expressive than emoticons since they can function as both
punctuating devices and potentially units of larger meaning” (Markman/Ōshima 2007: 14).
As Katsuno and Yano (2007) explain, kaomoji feature a quite complex structure and may incorpo-
rate words, movements and onomatopoeic sounds like single frames of manga. Such a complex-
ity allows kaomoji to stand on their own and to express meanings working better than the verbal
code.
The rst set of emojis directly stems out of the discursive tradition of Japanese emoticons,
since, as we have seen, Shigetara Kurita developed them in order to facilitate the otherwise com-
plex creation of kaomoji symbols on keyboards and thus overcome the limits imposed by the
modal affordances of mobile phones (Adami/Kress 2010).
Their use as narrative instruments is evident in many examples such as the already quoted Ben-
enson’s ‘emoji ed’ version of Moby Dick or the emoji translation of President Obama’s State of
the Union Address 2015 published by The Guardian-US (2015). However, the analyses of Chiusa-
roli (2015) Nasser, Benenson et al. (2013) and Stark and Crawford (2015) show that, when emojis
are used in global digital discourse (that is not in the speci c cultural context out of which they
originated), they are able to convey common and ‘universal’ meanings, while their full seman-
tic and pragmatic realization is highly dependent on the presence of a verbal co-text/context.
7. Conclusions
The aim of the present article was the decoding of the semantic and semiotic layers of the “Face
with Tears of Joy” emoji, which was chosen by the Oxford Dictionaries as the “Word of the
Year” 2015. In order to map the history of the sign, a socio-semiotic perspective has been ad-
opted and the concept of translation as “transduction” has been used to trace the movement of
meaning across sign systems, with a special critical focus on the cultural contexts out of which
they originated.
The author maintains that this ‘pictographic word’ is to be seen as a marker of the merging of
Japanese and American cultures in the discursive practices of geek communities. In particular, she
argues that such a sign can be de ned a Japan-America mash-up since it presents traces of the
two different cultural traditions as well as of the subsequent waves of transpaci c exchanges.
Hermes-55-moschini.indd 21 26-07-2016 13:29:56
22
Indeed, the “Face with Tears of Joy” that was selected by the Oxford Dictionaries is Apple’s pro-
prietary design (which is the de-facto standard of contemporary digital discourse) and presents the
fusion of manga visual style and Japanese kaomoji with the tradition of pin-button 1960s smiley
faces celebrating the optimistic commercial ethos of US capitalism. The intertwining of the two
cultural traditions has occurred primarily thanks to the exchanges of the discursive practices of
computer culture communities, which are deeply intertwined with the practices of geek com-
munities such as sci- , fantasy and comics fandom, now become popular with the spreading of
digital discourse.
As a matter of fact, the ow of cultural and subcultural material between America and Japan
has created a global postmodern culture where aesthetic, artistic, and intellectual traditions in u-
ence each other across national boundaries (Murakami 2005, Kelts 2006, Tatsumi 2006). In par-
ticular, such transcultural processes have given rise to a ‘pop-cultural fanaticism’ that has now
become mainstream because of a key generation of people ‘raised on a diet’ of transpaci c cultu-
ral products (Rivera 2008: 137).
It appears thus that the Oxford Dictionaries’ word of the year here analyzed being – as it is – an
example of such an intertextual magma, could be considered a potential component of the “data-
base fantasyscape”, the virtual ‘repository’ theorized by Brian Ruh (2014) that, merging Azuma’s
database concept (2007, 2010 [2001]) and Napier’s idea of fantasyscape (2007), aims to descri-
be the ow of anime and manga in a transmedia and transnational context. Such an interpretation
would conceptualize emojis, like anime and manga characters, as just another global postmodern
‘tile’ people may gather in order to create their own DIY narrations.
In addition, the transduction of Kurita’s emojis to the Unicode encoding system and the cre-
ation of the new global standard for the Japanese signs suggest a process of ‘universalization’
which is deeply imbued with American cultural values, a feature that appears to be distinctive
of digital discourse (Moschini 2013).
Acknowledgements
I would like to thank Dr Gianluca di Fratta for his precious supervision on Japanese culture. Many
thanks also go to the reviewer who invited me to read Brian Ruh’s work on the database fanta-
syscape.
References
(All the digital sources were last accessed in December 2015)
Adami, Elisabetta/Kress, Gunther 2010: The Social Semiotics of Convergent Mobile Devices: New Forms of Com-
position and the Transformation of Habitus. In Kress Gunther (ed.), Multimodality. A Social Semiotic Approach to
Contemporary Communication. London & New York: Routledge, 184-197.
Allen, Erin 2013: A Whale of an Acquisition. In Library of Congress Blog, February 22 [online]. http://blogs.loc.gov/
loc/2013/02/a-whale-of-an-acquisition
Azuma, Hiroki 2007: The Animalization of Otaku Culture. In Mechademia 2, 175-187.
Azuma, Hiroki 2010: Generazione Otaku. Uno studio della postmodernità. Milano: Jaca Book. [Dōbutsuka suru Posu-
tomodan. Otaku kara mita Nihon shakai. Tōkyō: Kōdansha 2001].
Badgeomania 2014: How the Button Was Born, February 6 [online]. https://www.facebook.com/notes/badgeomania/
how-the-button-was-born-us-patent-564356-pinback-button-badge-history/541935432568735?fref=nf
Baron, Naomi S. 2000: Alphabet to Email: How Written English Evolved and Where It’s Heading. London/New York:
Routledge.
Baron, Naomi S. 2008: Always On: Language in an Online and Mobile World. Oxford & New York: OUP.
Becker, Joseph D. 1998 [1988]: Unicode 88. Mountain View, CA: Unicode Consortium Press [online]. http://www.
unicode.org/history/unicode88.pdf
Belting, Hans 1990: The Image and its Public in the Middle Ages. New Rochelle, NY: Aristide D. Caratzay.
Benenson, Fred (ed) 2010: Emoji Dick. New York: Harper-Collins Publishers.
Hermes-55-moschini.indd 22 26-07-2016 13:29:56
23
Benford, Gregory 1996: alt.fans. The Internet is Recapitulating Science Fiction Fandom. In Reason, January, 1 [online].
https://reason.com/archives/1996/01/01/altfans
Birren, Faber 1961: Color Psychology and Color Therapy. Secaucus, NJ: University Books.
Blagdon, Jenn 2013: How Emoji Conquered the World. The Story of the Smiley Face from the Man Who Invented it. In
The Verge, March 4 [online]. http://www.theverge.com/2013/3/4/3966140/how-emoji-conquered-the world
Borodo, Michal 2015: Multimodality, Translation and Comics. In Perspectives. Studies in Translatology 23 (1), 22-41.
Bouissou, Jean-Marie 2011 [2010]: Il Manga. Storia e Universi del Fumetto Giapponese, Latina, Tunué Editori dell’im-
maginario [Manga. Histoire et univers de la bande dessinée japonaise, Arles, Édition Philippe Picquier, 2010].
Brand, Stewart 1972: Spacewar: Fanatic Life and Symbolic Death among the Computer Bums. In Rolling Stone, De-
cember 7.
http://digitizingamerica.shanti.virginia.edu/sites/shanti.virginia.edu.digitizingamerica/ les/SPACEWAR%20-%20
by%20Stewart%20Brand.pdf
Chiusaroli, Francesca 2015: La scrittura in Emoji tra dizionario e traduzione. In Bosco, Cristina/Zanzotto, Fabio M./
Tonelli, Sara (eds.), Proceedings of the Second Italian Conference on Computational Linguistics – Trento, Dicem-
bre. Torino: Accademia University Press, 88-93.
Cohn, Neil 2010: Japanese Visual Language: The Structure of Manga. In Johnson-Woods, Toni (ed.), Manga: An An-
thology of Global and Cultural Perspectives. New York: Continuum Books, 187-203.
Crystal, David 2001: Language and the Internet. Cambridge, UK: Cambridge University Press.
Davis, Mark 2014: Unicode Emoji – Past, 38th Internationalization and Unicode Conference Keynote Speech [online].
https://www.youtube.com/watch?v=-n9ONNeACyw
Davis, Mark/Edberg, Peter 2015: Unicode Emoji – Unicode® Technical Report #51 [online]. http://www.unicode.org/
reports/tr51/index.html
Desjardins, Renée 2008: Inter-Semiotic Translation and Cultural Representation within the Space of the Multi-Modal
Text. In TranscUlturAl 1 (1), 48-58.
Dresner, Eli/Herring, Susan C. 2012: Emoticons and Illocutionary Force. In Dana Riesenfel/Scara le Giovanni (eds.),
Philosophical Dialogue: Writings in Honor of Marcelo Dascal. London: College Publication, 59-70.
Gambier, Yves 2006: Multimodality and Audiovisual Translation. In Carroll, Mary/Gerzymisch-Arbogast, Heidrun/
Nauert, Sandra (eds.), MuTra 2006 – Audiovisual Translation Scenarios: Conference Proceedings. Copenhagen:
MuTra [online]. http://euroconferences.info/proceedings/2006_Proceedings/2006_Gambier_Yves.pdf
Gorlé e, Dinda L. 1994: Semiotics and the Problem of Translation: With Special Reference to the Semiotics of Charles
S. Peirce (Approaches to Translation Studies). Amsterdam/Atlanta (GA): Rodopi.
Hjorth, Larissa 2009: The Art of Being Novel: Rethinking Cartographies of Personalization. In Cubitt Sean/Paul Thom-
as (eds.), Re: Live – Third International Conference on the Histories of Media Art, Science and Technology Pro-
ceedings. Melbourne: The University of Melbourne Press, 65-72 [online]. http://www.mediaarthistory.org/relive/
ReLive09Proceedings.pdf
Hodge, Robert/Kress, Gunther 1993: Language as Ideology. London/New York: Routledge.
Iedema, Rick A.M. 2001: Resemiotization. In Semiotica 137, 1(4), 23-39.
Isaacson, Walter 2014: The Innovators. How a Group of Hackers, Geniuses and Geeks Created the Digital Revolution.
New York: Simon & Schuster.
Jakobson, Roman 2004 [1959]: On Linguistic Aspects of Translation. In Venuti, Lawrence (ed.), The Translation Stud-
ies Reader. London/New York: Routledge, 138-43.
Jenkins, Henry 2013 [1992]: Textual Poachers: Television Fans and Participatory Culture (Studies in Culture and
Communication). New York: Routledge.
Jewitt, Carey 2009: An Introduction to Multimodality. In Jewitt, Carey (ed.), The Routledge Handbook of Multimodal
Analysis. London: Routledge, 11-27.
Katsuno, Hirofumi/Yano, Christine R. 2002: Face to Face: On-line Subjectivity in Contemporary Japan. In Asian Stud-
ies Review 26 (2), 205-231.
Kelts, Roland 2006: Japanamerica. How Japanese Pop Culture Has Invaded the US. New York: Palgrave MacMillan.
Kinsella, Sharon 1995: Cuties in Japan. In Skov, Lise/Moeran, Brian (eds.), Women, Media and Consumption in Ja-
pan. London: Curzon Press, 220-254.
Konzack, Lars 2014: The Origins of Geek Culture: Perspectives on a Parallel Intellectual Milieu. In Bowman, Sarah L.
(ed.), Wyrd Con Companion Book. Costa Mesa: WyrdCon 4, 52-59.
Kress, Gunther 1997: Before Writing: Rethinking the Paths to Literacy. London/New York: Routledge.
Hermes-55-moschini.indd 23 26-07-2016 13:29:56
24
Kress, Gunther 2000: Text as the Punctuation of Semiosis: Pulling at Some Threads. In Meinhof, Ulrike H./Smith, Jon-
athan M. (eds.), Intertextuality and the Media: from Genre to Everyday Life. Manchester: Manchester UP, 132-154.
Kress, Gunther/Jewitt, Carey 2003: Introduction. In Kress, Gunther/Jewitt, Carey (eds.), Multimodal Literacy. New
Literacies and Digital Epistemologies. New York: Peter Lang, 1-18.
Kress, Gunther/van Leeuwen, Theo 2001: Multimodal Discourse: The Modes and Media of Contemporary Communi-
cation. London: Hodder Arnold.
Kress, Gunther/van Leeuwen, Theo 2006 [1996]: Reading Images. The Grammar of Visual Design. London/New York:
Routledge.
Krohn, Franklin 2004: A Generational Approach to Using Emoticons as Non-Verbal Communication. In Journal of
Technical Writing and Communication, 43, 321-328.
Lovering, Daniel 2007: Digital ‘Smiley Face’ Turns 25 :-). In Live Science, September 18 [online]. http://www.li-
vescience.com/4604-digital-smiley-face-turns-25.html
Markman, Kris M./Ōshima, Sae 2007: Pragmatic play? Some Possible Functions of English Emoticons and Japanese
Kaomoji in Computer-Mediated Discourse. Paper presented at the Association of Internet Researchers Annual Con-
ference 8.0: Let’s Play!. Vancouver, BC, Canada, October.
Moore, Alan/Gibbons, Dave/Higgins, John 1986-1987: Watchmen. Burbank (CA): DC Comics.
Moschini, Ilaria 2013: Liberty Icons: Linguistic and Multimodal Notes on the Cultural Roots of Digital Technologies.
In LEA – Lingue e Letterature d’Oriente e d’Occidente, 2, 537-552 [online]. http://dx.doi.org/10.13128/LEA-1824-
484x-1384
Munday, Jeremy 2012: Introducing Translation Studies. Theories and Applications. London/New York: Routledge
[Kindle Version].
Murakami, Takashi 2005: Little Boy: The Arts of Japan’s Exploding Subculture. New Haven (CT): Yale UP.
Nakano, Mamiko 2015: Why and How I Created Emoji, Interview with Shigetaka Kurita. In Ignition [online]. http://
ignition.co/105
Napier, Susan J. 2007: From Impressionism to Anime: Japan as Fantasy and Fan Cult in the Mind of the West. New
York: Palgrave Macmillian.
Nasser, Ramsey/Benenson, Fred/Salditch, Zoë/Wortham, Jenna/Weber, Lindsey (moderator) 2013: ‘I Have No Words’.
Emoji and the New Visual Vernacular. Panel Discussion at the Emoji Art & Design Show, New York, December 14
[online]. http://www.emojishow.com/video
Natsume, Fusanosuke/ Takekuma, Kentarō (1995). Manga to iu “kigou”. In M. Inoue (ed), Bessatsu Takarajima EX,
Manga no yomikata. Tōkyō: Takarajimasha, Inc., 73-124.
Negishi, Mayumi 2014: Meet Shigetaka Kurita, the Father of Emoji. In The Wall Street Journal/Japan, March 26 [on-
line]. http://blogs.wsj.com/japanrealtime/2014/03/26/meet-shigetaka-kurita-the-father-of-emoji/
OED 2015 Emoji entry [online]. http://www.oxforddictionaries.com/de nition/english/emoji
O’Sullivan, Carol/Jeffcote, Caterina 2013 (eds.): Translating Multimodalities. In Journal of Specialized Translation,
20, Special Issue.
“Oxford Dictionaries Word of the Year”, Oxford Dictionaries Blog, November 16 2015 [online]. http://blog.oxford-
dictionaries.com/2015/11/word-of-the-year-2015-emoji
Pellitteri, Marco 2008: Il Drago e la Saetta. Modelli, strategie e identità dell’immaginario giapponese. Latina: Tunué
Editori dell’Immaginario.
Piercy, Joseph 2013: Symbols: A Universal Language. London: Michael O’Mara Books [Kindle Version].
Raymond, Eric S. (ed) 1993: The New Hacker’s Dictionary. Cambridge (MA): MIT Press [online]. http://catb.org/~esr/
jargon/html/index.html
Rivera, Renato 2008: The Popularization of Geek Culture, and the Marginalization of Otaku Culture. In Journal of
Kyoto Seika University 34, 122-140.
Robb, Alice 2014: How Using Emoji Makes Us Less Emotional. And What Linguists Say it Means if Your Smiley Face
Has a Nose. In New Republic, July 7 [online]. https://newrepublic.com/article/118562/emoticons-effect-way-we-
communicate-linguists-study-effects
Ruh, Brian 2014: Conceptualizing Anime and the Database Fantasyscape. In Mechademia 9, 164-175.
Schodt, Frederik 1983: Manga! Manga! The World of Japanese Comics. New York: Kodansha USA Inc.
Schodt, Frederik 1994: America and the Four Japans. Friend, Foe, Model, Mirror. Berkeley, CA: Stone Bridge Press.
Schodt, Frederik L. 1996: Dreamland Japan: Writings on Modern Manga. Berkeley, CA: Stone Bridge Press.
Hermes-55-moschini.indd 24 26-07-2016 13:29:56
25
Snell-Hornby, Mary 2006: The Turns of Translation Studies: New Paradigms Or Shifting Viewpoints? Amsterdam/
Philadelphia: John Benjamins.
Stamp, Jimmy 2013: Who Really Invented the Smiley Face?. In The Smithsonian Blog, March 13 [online]. http://www.
smithsonianmag.com/arts-culture/who-really-invented-the-smiley-face-2058483/?no-ist
Stark, Luke/Crawford, Kate 2015: The Conservatism of Emoji: Work, Affect, and Communication. In Social Media +
Society July-December 1 (2), 1-11.
Stecconi, Ubaldo 2010: Translation Studies. In Gambier, Yves/ van Doorslaer, Luc (eds.), The Handbook of Translation
Studies. Amsterdam: John Benjamins, vol. 1, 314-319.
Steinmetz, Katy 2015: Oxford’s 2015 Word of the Year Is This Emoji. Time, November 16 [online]. http://time.
com/4114886/oxford-word-of-the-year-2015-emoji
Sugimoto, Taku/Levin James A. 2000: Multiple Literacies and Multimedia: A Comparison of Japanese and American
Uses of the Internet. In Hawisher Gail E./Selfe Cynthia L. (eds.), Global Literacies and the World Wide Web. Lon-
don: Routledge, 133-153.
Tatsumi, Takayuki 2006: Full Metal Apache. Transactions Between Cyberpunk Japan and Avant-Pop America. Dur-
ham and London: Duke UP.
Taylor, Christopher 2004: Multimodal Text Analysis and Subtitling. In Ventola, Eija/Cassily, Charles/Martin Kalten-
bacher (eds.), Perspectives on Multimodality. Amsterdam/Philadelphia: John Benjamins, 153-172.
Taylor, Insup/Taylor, Maurice Martin 1995: Writing and Literacy in Chinese, Korean, and Japanese. Amsterdam/
Philadelphia: John Benjamins.
The Guardian – US, 2015, January 20: State of the Union … in Emojis [online]. http://www.theguardian.com/us-news/
ng-interactive/2015/jan/20/-sp-state-of-the-union-2015-address-obama-emoji
Torop, Peeter 2000: Intersemiosis and Intersemiotic Translation. In European Journal for Semiotic Studies 12(1), 71-
100.
Turner, Fred 2006: From Counterculture to Cyberculture. Stewart Brand, the Whole Earth Network and the Rise of
Digital Utopianism. Chicago (IL): University of Chicago Press.
Tymoczko, Maria 2005a: Enlarging Translation Theory: Integrating Non-Western Thought about Translation. In Her-
mans, Thomas (ed.), Translating Others. Manchester: St. Jerome, 13-32.
Tymoczko, Maria 2005b: Trajectories of Research in Translation Studies. In Journal des Traducteurs / Meta: Transla-
tors’ Journal 50 (4), 1082-1097.
Unicode Consortium 2015a: Full Emoji Data [online]. http://unicode.org/emoji/charts/full-emoji-list.html
Unicode Consortium 2015b: What is Unicode? [online]. http://www.unicode.org/standard/WhatIsUnicode.html
van Leeuwen, Theo 2011: The Language of Colour. An Introduction. London/New York: Routledge.
van Leeuwen, Theo 2004: Introducing Social Semiotics. London/New York: Routledge.
Wallestad, Thomas J. 2013: Developing the Visual Language of Comics: the Interactive Potential of Japan’s Contribu-
tion. In Expressive Culture 7, 3-13.
Walther, Joseph P./D’Addario, Kyle B. 2004: The Impacts of Emoticons on Message Interpretation in Computer-Medi-
ated Communication. In Social Science Computer Review 19, 324-347.
Hermes-55-moschini.indd 25 26-07-2016 13:29:56
... This is a re-framing of an old question in sociolinguistics, which is, as Kress puts it, 'how does this signifying object work here' (Kress, 2010: 1). Emoji have a rich socio-semiotic history, which creates a complex domain of potential meanings (Moschini, 2016;Sampietro, 2016a): they were first developed in Japan in the mid-1990s where a telecom company created them as a new communication form for pager users. Since then, the Unicode consortium 2 instantiated a set of standardised characters that form a cross-industry coding standard for emoji representation. ...
... Moschini suggests that emoji represent "a marker of the mashing up of Japanese and American cultures in the discursive practices of geek communities, now gone mainstream..." (Moschini, 2016: 9). Emoji signification has roots in the semiotics of Japanese manga cartoons, American 1960s counter-culture, the 1980s/90s acid house movement and, before all of that, an internal marketing campaign of a life insurance company in Massachusetts (Danesi, 2016;Moschini, 2016;Stark and Crawford, 2015). This paper focusses on analysing the face covering hand emoji. ...
Article
This paper uses conversation analysis to explore the communicative functions of one emoji in a mobile reading community in China. In contrast to semiotic approaches to emoji that focus on their cultural signification, or that treat them as reflections of users’ inner intensions, we analyse emoji as communication phenomena by exploring their relation to other textual actions in the production of text-talk. The emoji analysed here functioned as a laughter token, and performed specific interactional work related to laughter. We conclude that conversation analysis offers an important corrective to abstracted semiotic analysis and a useful resource for exploring the demonstrable meaning of emoji for interlocutors. However, we also emphasise the importance of capturing the process of composing messages, the challenges of dealing with the variety of forms that emoji take and their relation to gestural and other actions in face to face communication.
... In the late 1990s, a set of pictorial signs were then created in Japan deemed 'emoji'-picture (e) character (moji)-which became integrated into digital keyboards. Many emoji were pulled from the conventional depictions of emotions used in Japanese manga (Moschini, 2016), as evident in the nose bubble for sleep , shade on the forehead for dread , and the large sweat drop on the forehead for anxiety (Cohn, 2013a;Cohn & Ehly, 2016). As people began using emoji, emoticons reduced in prevalence (Pavalanathan & Eisenstein, 2016) or became automatically converted into emoji, and now emoji have become a common part of interactive digital communication. ...
Article
Full-text available
Abstract Emoji have become a prominent part of interactive digital communication. Here, we ask the questions: does a grammatical system govern the way people use emoji; and how do emoji interact with the grammar of written text? We conducted two experiments that asked participants to have a digital conversation with each other using only emoji (Experiment 1) or to substitute at least one emoji for a word in the sentences (Experiment 2). First, we found that the emoji-only utterances of participants remained at simplistic levels of patterning, primarily appearing as one-unit utterances (as formulaic expressions or responsive emotions) or as linear sequencing (for example, repeating the same emoji or providing an unordered list of semantically related emoji). Emoji playing grammatical roles (i.e., ‘parts-of-speech’) were minimal, and showed little consistency in ‘word order’. Second, emoji were substituted more for nouns and adjectives than verbs, while also typically conveying nonredundant information to the sentences. These findings suggest that, while emoji may follow tendencies in their interactions with grammatical structure in multimodal text-emoji productions, they lack grammatical structure on their own.
... According to the study about Instagram, emojis are present in up to 57% of online messages in many countries 3 . For example, "face with tears of joy", an emoji that means that somebody is in an extremely good mood, was regarded as the 2015 word of the year by The Oxford Dictionary [12]. In our opinion ignoring emoticons in sentiment research is unjustifiable, because they convey a significant emotional information and play an important role in expressing emotions and opinions in social media [13,5]. ...
Conference Paper
Internet slang is an informal language used in everyday online communication which quickly becomes adopted or discarded by new generations. Similarly, pictograms (emoticons/emojis) have been widely used in social media as a mean for graphical expression of emotions. People can convey delicate nuances through textual information when supported with emoticons. Furthermore, we also noticed that when people use new words and pictograms, they tend to express a kind of humorous emotion which is difficult to clearly classify as positive or negative. Therefore, it is important to fully understand the influence of Internet slang and emoticons on social media. In this paper, we propose a machine learning method considering Internet slang and emoticons for sentiment analysis of Weibo, the most popular Chinese social media platform. In the first step, we collected 448 frequent Internet slang expressions as a slang lexicon, then we con- verted the 109 Weibo emoticons into textual features creating Chinese emoticon lexicon. To test the capability of recognizing humorous posts, we utilized both lexicons with several machine learning approaches, k-Nearest Neighbors, Decision Tree, Random Forest, Logistic Regression, Naive Bayes and Support Vector Machine for detecting humorous expressions on Chinese social media. Our experimental results show that the proposed method can significantly improve the performance for detecting expressions which are difficult to polarize into positive- negative categories.
Article
Full-text available
Writing is considered a very important learning tool for all content areas. There is no doubt that writing skill is essential for any kind of learning either at the school or the university level. Therefore, the present study attempted to explore the possibility of improving university students’ one of writing sub-skills, known as coherence through emoji. Coherence in the students’ writing was measured according to the students ability to a) use repetition to link ideas, sentences, and paragraphs, b)use transitional expressions to link ideas, sentences, and paragraphs, c) use pronouns to link sentences, d)use synonyms to link ideas and create variety, e) use parallel structures to link ideas, sentences, and paragraphs. The sample of this study was randomly selected from the Third Level female students joining the Department of English at one Saudi public university. The selected subjects were assigned randomly into control group and experimental group. After administrating the pretest for the two groups, the experimental group was given a six-week treatment of emoji meant for improving the writing sub-skill, while the control group was taught writing without using emoji. One week after the treatment, a posttest was administered to the two groups. Results revealed significant difference between the students’ scores in writing in favor of using emoji to improve students’ writing skills. Thus, the findings show the effectiveness of using emoji in improving students’ coherence in writing. The results were used to draw some conclusions and to suggest some recommendations for teaching writing using emoji. The results also recommend further researches to be done on using emoji to improve other language skills and sub-skills.
Article
Among the pictorial representations that have succeeded in electronic-mediated communication (EMC), the rise of emoji is of special interest, as they have widely spread around the world. This article reports on the results of a survey conducted among students and staff of a large Spanish university (63% women) on the use of emoticons and emoji and their interpretation. Results confirm the preference for emoji over emoticons. The responses also disclosed emerging norms on the use and functions of these pictographs in EMC: Emoji are considered more “expressive” than emoticons, and messages without these cues are judged as “rude.” Regarding the interpretation of emoji, the survey shows that it heavily relies on preexisting iconography, even if new visual conventions are developing.
Conference Paper
Nowadays, social media have become the essential part of our lives. Internet slang is an informal language used in everyday online communication which quickly becomes adopted or discarded by new generations. Similarly, pictograms (emoticons/emojis) have been widely used in social media as a mean for graphical expression of emotions. People can convey delicate nuances through textual information when supported with emoticons, and the effectiveness of computer-mediated communication is also improved. Therefore, it is important to fully understand the influence of Internet slang and emoticons on social media. In this paper, we propose a convolutional neural network model considering Internet slang and emoticons for sentiment analysis of Weibo which is the most popular Chinese social media platform. Our experimental results show that the proposed method can significantly improve the performance for predicting sentiment polarity.
Article
The phenomenon of emojis has had many implications for the future course of writing, literacy, communications, and the nature of representation itself. This paper looks at the implications of emoji use through the filter of Saussurean semiotics and through the lens of theories of visuality, which claim that visual writing is having radical effects on literacy and cognition. The historical background to the rise of visual writing is used as a backdrop to the semiotic analysis of the emoji phenomenon. The way we read and write messages today with visual elements such as emoji may indicate a radical shift away from a linear mode of processing information, as imprinted in alphabetic forms of writing, toward a more holistic and imaginative mode. However, because emoji usage and creativity depend on specific technologies, it remains to be seen if such writing can survive as technologies change. The main argument in this paper is that emojis are more part of parole than they are a separate langue , but they nonetheless reveal changes that the latter is undergoing in an age of digital multimodal communication.
Chapter
In the new age of digital communication, when most of our human-to-human interaction takes place on the internet, new strategies to make up for the lack of non-verbal cues such as body language or voice tone, needed to be invented. This, in turn, led to the popularization of digital pictograms and ideograms, which have evolved greatly over the past two decades, allowing people to express themselves with a new type of non-verbal cue. Digital pictograms and ideograms developed into the virtual “body language” of the digital age, which, at first glance, might appear understandable without knowing the interlocutor’s language or the background to it. As the phenomenon spread around the world and across different cultures, it became clear that some digital pictograms and ideograms differ. In particular, differences between Western and Eastern-style pictograms became a focus for researchers. However, even different ‘Eastern-style emoticons’ may create confusion among the users. Two countries known for their extensive use of digital pictograms and ideograms in everyday communication are Japan and Korea. Their cultures are often compared, and although they share numerous characteristics, there are some important linguistic differences between them that should not be overlooked. Of particular interest here are the differences resulting from diversity in digital communication between Japan and Korea and these have not been paid the attention they deserve. This research aims to address the problem of cultural bias (by users from seemingly similar backgrounds) in interpreting and understanding ‘Eastern-style emoticons’. Using a specially designed survey (a mix of forced choice and open questions), we investigate main differences in the reception of keurim mal in Korea and kaomoji in Japan. Moreover, we attempt to anticipate problems which may emerge from possible misunderstandings (or misinterpretations) of such.
Article
Full-text available
Though Jakobson conceptualized inter-semiotic transfer as a valid form of translation between texts some 40 years ago, it remained a relatively marginal area of investigation until recently. Additionally, we may note that information and communication technologies (ICTs) have also markedly modified previously held notions of what constitutes a “text”. Basing my research on these two observations, this paper will draw attention to inter-semiotic translation within the space the newscast, a prime example of an ever-evolving multi-modal text created by newer forms of ICTs. More specifically, the focus lies in how we can posit translational activity in the construction of the newscast (for instance, how the “visual” images translate the “verbal” narration of the newscaster or journalist). This conceptualization of “news-making” will lead us to consider the ways in which cultural stereotypes are created through the “translation” of interacting texts (verbal, visual, aural) on the same interface. To suggest that newscasts, and by extension media, proliferate cultural stereotypes, is by no means novel. However, to consider how inter-semiotic translation plays a role in their creation may be a departure from previous paradigms. In fact, Kress and van Leeuwan state: “This incessant process of ‘translation’, or ‘transcoding’ – ‘transduction’ – between a range of semiotic modes represents […] a better, more adequate understanding of representation and communication (2006:39)”. Furthermore, because cultural stereotyping is often at the root of conflict, this type of investigation becomes, we suggest, all the more worthwhile in understanding how we “translate” difference across borders, semiotic or otherwise.
Article
Full-text available
Emoticons are graphic representations of facial expressions that many e-mail users embed in their messages. These symbols are widely known and commonly recognized among computer-mediated communication (CMC) users, and they are described by most observers as substituting for the nonverbal cues that are missing from CMC in comparison to face-to-face communication. Their empirical impacts, however, are undocumented. An experiment sought to determine the effects of three common emoticons on message interpretations. Hypotheses drawn from literature on nonverbal communication reflect several plausible relationships between emoticons and verbal messages. The results indicate that emoticons' contributions were outweighed by verbal content, but a negativity effect appeared such that any negative message aspect-verbal or graphic-shifts message interpretation in the direction of the negative element.
Chapter
Full-text available
For some years now this author has collaborated with colleagues at the universities of Pavia, Padua and Venice in research projects concerning multimodal text analysis and the creation of multimodal corpora, e.g. Taylor (2000). In particular the work of Thibault (2000) and Baldry (2000) has inspired work on the harnessing of multimodal transcriptions to the task of translating screen texts for interlingual subtitles. Specific translation (and other) strategies are required in subtitling film scripts and the like, and the multimodal transcription provides a scientific basis for formulating some of those strategies which are involved in the particularly controversial process of text condensation. A whole series of different multimodal text types have been analysed (feature films, soap operas, advertisements, cartoons, documentaries, etc.) and this chapter will illustrate the methodology underlying and the results obtained from a number of those analyses.
Article
Full-text available
This piece examines emoji as conduits for affective labor in the social networks of informational capitalism. Emoji, ubiquitous digital images that can appear in text messages, emails, and social media chat platforms, are rich in social, cultural, and economic significance. This article examines emoji as historical, social, and cultural objects, and as examples of skeuomorphism and of technical standardization. Now superseded as explicitly monetized objects by other graphics designed for affective interactions, emoji nonetheless represent emotional data of enormous interest to businesses in the digital economy, and continue to act symbolically as signifiers of affective meaning. We argue that emoji characters both embody and represent the tension between affect as human potential, and as a productive force that capital continually seeks to harness through the management of everyday biopolitics. Emoji are instances of a contest between the creative power of affective labor and its limits within a digital realm in the thrall of market logic.
Article
In spite of the growing body of literature on anime, very few people have examined what the term denotes. Most simply equate “anime” with “Japanese animation”; this is a relatively easy “out” in order to get past tricky questions of definition, and one to which I must admit engaging in on occasion. For example, in Stray Dog of Anime: The Films of Mamoru Oshii, I wrote that “‘anime’ does not denote any particular style or content; it simply means animation from Japan.” Carl Silvio has rightly criticized me for this overly simplistic statement, asserting that “Ruh offers us a definition [of anime] that is virtually invulnerable to refutation.” This definition presents something of a problem because many fans would not consider all animation from Japan as anime. For example, when Yamamura Kōji’s Mt. Head (2002, Atamayama) was nominated for an Academy Award for best animated short film in 2003, it was relatively ignored by “anime” fans. The film’s entry in the online encyclopedia on the Anime News Network website even states “Some people would not consider this as Anime [sic], but rather ‘Alternative Japanese Animation.’” This entry goes into no further detail and does not explain how “Alternative Japanese Animation” differs from “anime,” but it seems clear that some fans have drawn a line around the concept of “anime” that does not include all animation from Japan. A useful tool for examining the definitional limbo of the term “anime” is Matt Hills’s study of fan cultures. In his book’s preface, Hills explicitly states that he is taking a “suspensionist” position in which he is “approaching the contradictions of fan cultures and cult media as essential cultural negotiations that can only be closed down at the cost of ignoring fandom’s cultural dynamics.” Like Hills, it is not my intention to arrive at a singular “expert” definition of anime but to pursue how anime is discussed within English-language fan and academic circles in order to explore how the meanings of such terms may change over time, sometimes taking on and discarding new connotations. However, even within this suspensionist viewpoint, “anime” cannot mean just anything. In order for the term to be meaningful, it needs to be given structure, even if this structure does not point in the end to a singular definition. Before the term “anime” came into widespread use in the late 1980s and early 1990s, such animation had often been called “Japanimation” in English. Anime historian Fred Patten traces the first use of the word “Japanimation” to around 1978, shortly after anime fandom became formalized with the establishment of the first U.S. anime fan club, called the Cartoon/Fantasy Organization (CFO), in May 1977. The term “Japanimation” began to fall from popularity nearly a decade later—one reason was that it got too easily twisted into a racially derogatory term by detractors (e.g. “Jap animation”). Most people now writing use the term “anime,” save a few. Japanese media scholar Ueno Toshiya, for instance, defends his use of the term “Japanimation,” saying, “I use this term to emphasize both geography and the particularity of its characteristic styles, for these are quite different from animations in the general sense.” In this way, such animation becomes territorially marked by the term “Japanimation”; “anime,” on the other hand, does not necessarily carry such connotations and seems to have the potential to be a much more fluid marker. Some books on anime have introduced their topic of discussion in these ways: “Japanese cartoons (more properly, anime, which simply means ‘animation’)”; “anime (Japanese animation)”; and “Japanese animation, or ‘anime,’ as it is now usually referred to in both Japan and the West.” Other writers take a different approach to examining anime. In his introduction to the edited collection Cinema Anime, Steven T. Brown writes, “What is anime? Anime is so multifarious in its forms and genres, its styles and audiences, that one needs to pose the question differently: Where is the anime screen?” Brown’s reformulated question suggests that the meaning of anime comes about through the process of viewing anime onscreen, wherever that may be, and, as such, the term “anime” is situationally dependent. In this...