How to export Mac's Address Book contents to simple kind-of-human-readable dump

Recently I wanted to take a dump of Mac’s Address Book contents. Just the names & addresses of my contacts in a human-readable format.

Some background

First, my Address Book version is 5.0.3. It has the functionality File → Export -> Address Book Archive. This command generates a directory that contains a file with .abcddb postfix that is a SQLite file. (In my case, the file name was AddressBook-v22.abcddb)

After that, I found the blog post AddressBook entries from a Mac using SQLite that has the needed SQL query with the columns and the joins sorted out.

To get the output to a file, a couple of SQLite commands were found from How to export the results of my query to csv file.

The result

The end result to be run with sqlite3 (a SQLite command-line utility). For all the possible columns, check the blog post linked above.

.headers on
.mode csv
.output export.csv

SELECT DISTINCT
ZABCDRECORD.ZFIRSTNAME [FIRST NAME],
ZABCDRECORD.ZLASTNAME [LAST NAME],
ZABCDPOSTALADDRESS.ZSTREET [STREET],
ZABCDPOSTALADDRESS.ZZIPCODE [ZIPCODE],
ZABCDPOSTALADDRESS.ZCITY [CITY]

FROM ZABCDRECORD
LEFT JOIN ZABCDPOSTALADDRESS on 
  ZABCDRECORD.Z_PK = ZABCDPOSTALADDRESS.ZOWNER
LEFT JOIN ZABCDNOTE ON 
  ZABCDRECORD.Z_PK = ZABCDNOTE.ZCONTACT

ORDER BY
ZABCDRECORD.ZLASTNAME,
ZABCDRECORD.ZFIRSTNAME;


.output stdout

Discussion

Note that entries containing newlines might need some cleanup (google for “sqlite replace newline”, for example) but that wasn’t needed in my case.

Lean Mindset

Lean Mindset

Some time ago I read the book The Lean Mindset: Ask the Right Questions by Mary & Tom Poppendieck.

The book isn’t very focused (either on Lean thinking on traditional sense or software development) but the book is a collection of interesting thoughts, stories & case studies related to lean thinking in a more broad scale. If you want an intro to Lean thinking on traditional sense, I recommend you to check out Lean Primer by Craig Larman and Bas Vodde.

I’ve collected here some interesting thoughts from or related to the book.

Two mindsets (dual processing theory: fast-thinking self vs. analytical part)

Dual processing theory is a model according to which our mind can be in two different modes: System 1 (Fast-thinking self, “auto-pilot mode”) & System 2 (Analytical part, slow)

• Keith Stanovich & Richard West: Dual processing theory
• Description can be found from Daniel Kahneman’s book Thinking, Fast and Slow

Douglas McGregor’s Theory X & Theory Y

• Theory X assumes that people don’t like work and will do as little as possible
• Theory Y assumes that most people are eager to work and want to do a good job
• Lean principle of Energized Workers is based on Theory Y
• Both theories can be self-fulfilling prophecies
• http://www.mindtools.com/pages/article/newLDR_74.htm

Full potential – Growth mindset etc.

• Garfield high school & ganas
  • Jaime Escelante & Jack Dirmann: The Jaime Escelante Math Program
• Fixed mindset vs. Growth mindset (Carol Dweck: Mindset)
  • Fixed mindset: Talent is something that people either have or don’t have. (I people conclude they aren’t good at something, they abandon it quickly)
  • Growth mindset: Talent is not fixed, with hard work you can become at whatever you choose.
  • Check out Derek Sivers’ blog: Fixed mindset vs Growth mindset
• Expertise development, for example musicians and athletes:
  • Challenge: They decide to pursue a challenging goal.
  • Coaching: They practice under the guidance of a coach.
  • Progress: They push their limits, fail, and receive immediate feedback.
  • Perseverance: They repeat this over and over again, many times a day, for years
• Troy Higgins’ Regulatory Fit Theory: Two kinds of people (take with a grain of salt)
  • People with promotion focus, kind of optimists
    • Aspirations are strong motivators
    • Take-the-company-to-next-level
  • People with prevention focus, kind of pessimists
    • Meeting obligations and fear of failure are strong motivations
    • Save-the-company-from-disasters
  • More at http://www.columbia.edu/cu/psychology/higgins/research.html#rfit
• Intuition
  • Daniel Kahnemann and Gary Klein, Conditions for Intuitive Expertise
  • Good to note Cognitive Biases
  • Gary Klein, how people do decisions

Delighted Customers

• Quote: The Westerner and the Japanese man mean something different when they talk of “making a decision.” In the West, all the emphasis is on the answer to the question. To the Japanese, however, the important element in decision making is defining the question. (Peter Drucker’s What We Can Learn from Japanese Management)
• Ask the right questions, solve the right problems (check out the book Are your lights on)
• Case Sphere of influence, Concept of Strong centers
  • Mixing Product Design/Innovation and Agile
  • Strong center to express the aspect of a products identity and wholeness
• Procter & Gamble approach to scientific planning, check out Bringing Science to the Art of Strategy
  • Frame a choice
  • Generate possibilities
  • Specify conditions
  • Identify barriers
  • Design tests
  • Conduct the tests
  • Make a choice

Design Toolbox – some tools

• Establish empathy
  • Journey map (check out for example Designing for Growth: a design thinking tool kit for managers)
  • Adaption chain analysis (Ron Adner: The Wide Lens: A new strategy for innovation)
• Generate possibilities
  • Press Release (Another possibility “fake ads”)
  • Impact Mapping
• Run experiments
  • Assumption testing
  • Rapid prototyping
  • Learning launch

How Spotify Builds Products

• Good stuff @ http://blog.crisp.se/2013/01/13/henrikkniberg/how-spotify-builds-products & https://dl.dropboxusercontent.com/u/1018963/Articles/HowSpotifyBuildsProducts.pdf

Disruptive innovations

• Success of established companies and products attracts competitors that often develop products that are less expensive, require less power, are smaller etc.
• Disruptive Technologies: Catching the Wave
• The Innovator’s Dilemma

Still some links :)

• IDEO Designing a shopping cart
  • Check also IDEO blog 7 tips on better brainstorming
• Transforming healthcare for children and their families: Doug Dietz at TEDx San Jose 2012

Advertising: A Very Short Introduction

This time I familiarized myself with domain of advertising with Advertising: A Very Short Introduction by Winston Fletcher. The book is part of Oxford University Press’ Very Short Introductions series. With this brief experience I could consider checking out also other titles from them. Some notes collected from the book follow, following the chapters of the book :)

1. Advertising

• Advertising – what is it?
  • Advertising is one type of marketing communication (other types include e.g. packaging, sales promotions, commercial emails, public relations etc.)
  • Definition by the writer: An advertisement is a paid-for communication intended to inform and/or persuade one or more people.
  • To sell isn’t always the (only) reason
• Advertising objectives for a campaign vary a lot. Some examples:
  • launch a new brand
  • launch a new product to an existing brand
  • make people who haven’t heard of a brand become aware of it
• Advertising strategy – this is where the campaign objectives should be spelled out
• Target market – defined population sectors
  • Target Group Index (TGI) survey in the UK
• What do customers want from their brand?
  • Emphasis has been shifted from product formulations towards end-benefits.
  • Brand image, emotional side
• Diversity of advertising – good to note
  • Diversity of services and products
• Account planners – usually the ones who study and analyze the wealth of information

2. Industry structure

Tripartite of advertisers, media and agencies

• Advertisers
  • Retailers, manufacturers, finance companies, charities, governments etc.
• Media
  • Print, TV, posters, radio, cinema and the Internet.
  • classifieds vs. display
    • Ads that people look for (classifieds)
    • Ads which look for people (display)
  • print & Internet have both classifieds & display
  • TV, radio, cinema and posters carry mainly display advertising
  • Before arrival of TV, print advertising was largely on classifieds
  • Digital revolution with the Internet
• Agencies
  • In the 1970’s, agencies have split to creative agencies and media agencies.
  • Digi revolution has resulted in digi departments and separate digi agencies.
  • This is the part of the tripartite that relies & works fully on advertising.

3. Advertisers

• Most businesses advertiser
  • Note that not all businesses advertise.
• 4 criteria for a (successful) brand:
  • The brand will have a unique name and packaging, and usually a logo, all of which will be easily and instantly recognizable by its target market.
  • It must be perceived by its target market to have qualities which differentiate it from others, similar brands. (brand positioning)
  • The qualities will be both functional and emotive; its purchasers will both think and feel the brand is different from, and in their eyes superior, others.
  • These perceptions will allow the brand to command a premium price over unbranded, commodity products, and thus to generate greater profits for the company that owns it.
• Valuation of the brand by its target market is what matters.

4. Media

• Media – Latin plural form of the singular noun medium
• Medium (Oxford English Dictionary): Any intervening substance through which a force acts on objects at a distance or through which impressions are conveyed to the senses.
• Criteria for choosing media
  • Reach – How many people in the target market does the medium reach ? What kind of people?
  • Cost – How much does it cost to use the medium? (Compared with alternative media, which provides the best value for money)
  • How powerful and persuasive it is as an advertising medium?
• Cost per thousand (readers/watchers/… in the target market) is the typical comparison criteria within certain type of media.
  • Note that it seldom is helpful to compare cost per thousand straight between media (effects with different types of media are too different)

Different types of media

• Note that the descriptions are from the UK point of view.

Press and Magazines

• National newspapers
• Regional newspapers
  • Higher cost per thousand than with nationals
• Consumer magazines
  • Higher reader-to-circulation ratio (one paper is usually read by more people)

TV

• In the UK, strong governmental control on TV advertising
• Fragmentation has increased recently
• Writer states that no evidence of the Internet or Video-on-demand resulting as TV viewership falling

Rest (minor media)

• Direct mail
• Outdoor and transport
• Radio
• Cinema

The Internet

• Has grown much
• Most Internet advertising for products & brands for which customers require detailed information.
• Provides much better options for targeting than other mediums

5. Creative agencies

• The messages you put into advertisements are but the means to an end.
• The end is what the target market makes of the messages.
• Copy/art cooperation
  • Before TV words were the beef → copywriter was the top
  • Nowadays working mostly as copy/art teams (copy & art together)
• Creativity is an incremental process
• Account planning & account management have a big role
• Globalization trend

6. Media agencies

• Advertising agencies have mostly been split to creative and media agencies (specialization)
• Media agencies: help advertisers to fish where the fish are – at the lowest possible cost
• In media agencies analysis, research, numbers are the thing
• Two roles: planning (strategic) & buying (negotiating)
• Media selection factors:
  • Budget
  • (Technical) nature of the medium
  • Coverage of the target market
• One principal expertise of media agencies is choosing between trade-offs
  • Media selection
  • Size, frequency, timing etc.
• Digital (Internet) has also brought pay-per-click charging

7. Research

• Some historical notes on advertising research
  • 1909: Psychology of advertising (Walter Dill Scott)
  • 1923: Scientific advertising (Claude Hopkins)
    • Still relevant stuff for coupon advertising
  • 1950’s: Motivation research
  • 1957: Subliminal advertising study (later admitted to have been a hoax)
  • from early 1940s to late 1960s: Unique Selling Proposition (Ted Bates)
• Today no more search for the Golden Key
• Pre-testing & post-launch testing
  • Pre-testing
    • Important that the interviewees are members of the target group
    • 1-1 or focus group interviews
  • Post-launch testing
    • Advertising awareness (only an approximation of campaign effectiveness)
    • KPIs (pre-defined)
• Other techniques have also been tried (pupil dilation, eye tracking, MRI, …)

Misc. links

• IPA Awards
• http://www.warc.com/

Are your lights on?

A while ago I happened to read DHH:s blog post The five programming books that meant most to me. Since the first four (Smalltalk Best Practice Patterns, Refactoring, Patterns of Enterprise Application Architecture and Domain-Driven Design) are all classics that I either have read or at least know of, I was curious to check out the fifth one that I haven’t heard of before, Are your lights on? How to figure out what the problem.

The book isn’t straightly about programming but the content is very relevant for anyone involved in creating software: Before starting to solve the problem, make sure you know what is the problem you’re trying to solve and is it the right problem at all. Whose problem is it?

To go through the book again I’ve collected a table-of-contents-ish summary of the chapters and main points of the book.

Part 1: WHAT IS THE PROBLEM?

1. A PROBLEM

• What is the problem?

Try to find

• Who has a problem?

and for each answering party, try to ask

• What is the essence of the problem?

2. PETER PIGEONHOLE PREPARED A PETITION

• How can we determine “What is wrong?”
• What is wrong?
• What can be done about it?

3. WHAT‘S YOUR PROBLEM?

• A problem is a difference between things as desired and things as perceived.
• Phantom problems are real problems.
  • Phantom problem: A discomfort primarily attributable to perceptions.

Part 2: WHAT IS THE PROBLEM?

4. BILLY BRIGHTEYES BESTS THE BIDDERS

• Don’t take their solution method for a problem definition.
• If you solve their problem too readily they’ll never believe you’ve solved their real problem.

5. BILLY BITES HIS TONGUE

• Don’t mistake a solution method for a problem definition – especially if it’s your own solution method.

6. BILLY BACK TO THE BIDDERS

• You can never be too sure you have a correct definition, even after the problem is solved.
• You can never be sure you have a correct definition, but don’t ever stop trying to get one.
• Don’t leap to conclusions, but don’t ignore your first impression.

Part 3: WHAT IS THE PROBLEM REALLY?

7. THE ENDLESS CHAIN

• Each solution is the source of the next problem.
• The trickiest part of certain problems is just recognizing their existence.
• If you can’t think of at least three things that might be wrong with your understanding of the problem, you don’t understand the problem.

8. MISSING THE MISFIT

• A misfit is a solution that produces a mismatch with the human beings who have to live with the solution.
  • Most misfits are easy to solve, once they are recognized.
• Don’t leap to conclusions, but don’t ignore your first impression.
• Test your definition on a foreigner, someone blind, or a child, or make yourself foreign, blind or childlike.
• Each new point of view will produce a new misfit.

9. LANDING ON THE LEVEL

• How could we change the problem statement to make the solution different?
• What am I solving?
• As you wander along the weary path of problem definition, check back home once in a while to see if you haven’t lost your way.

10. MIND YOUR MEANING

• Once you have a problem statement in words, play with the words until the statement is in everyone’s head.

Part 4: WHOSE PROBLEM IS IT?

11. SMOKE GETS IN YOUR EYES

• Whose problem is it?
• Don’t solve other people’s problems when they can solve them perfectly well themselves.
• If it’s their problem, make it their problem.

12. THE CAMPUS THAT WAS ALL SPACED OUT

• Whose problem is it?
• If a person is in a position to do something about a problem, but doesn’t have the problem, then do something so he does.
• Try blaming yourself for a change – even for a moment.

13. THE LIGHTS AT THE END OF THE TUNNEL

• Whose problem is it?
• If people really have their lights on, a little reminder may be more effective than your complicated solution.

Part 5: WHERE DOES IT COME FROM?

14. JANET JAWORSKI JOGGLES A JERK

• Where does this problem come from?

15. MISTER MATCZYSZYN MENDS THE MATTER

• Where does this discourtesy come from?
• The source of the problem is often within you.

16. MAKE-WORKS AND TAKE CREDITS

• Where does the problem come from?
• There are two kinds of people in the world…

17. EXAMINATIONS AND OTHER PUZZLES

• Where does the problem come from?
• Who sent this problem?
• What’s he trying to do to me?
• Puzzles are difficult by design

Part 6: DO WE REALLY WANT TO SOLVE IT?

18. TOM TIRELESS TINKERS WITH TOYS

• Most of us have had schooling – too much of it
  • → We’re developed an instinct that makes us seize upon the first statement that looks like a “problem”. (This strategy works well on exams etc.)
• In spite of appearances, people seldom know what they want until you give them what they ask for.

19. PATIENCE PLAYS POLITICS

• Not too many people, in the final analysis, really want their problems solved.

20. A PRIORITY ASSIGNMENT

• Do we really want a solution?
• We never have enough time consider whether we want it, but we always have enough time to regret it.
• The fish is always last to see water.

Summary

As a summary: We often are experienced problem solvers and thus pretty enthusiastic on solving problems. Because of that, before going to solve the problem, it is important to remember to stop to think what is the problem actually, whose is it and where does it come from. This book is a pretty fast read and good reminder on this.

It's Your Ship

I just finished reading It’s Your Ship by Captain D. Michael Abrashoff, a leadership book written by former captain of USS Benfold, a US Navy Guided missile destroyer.
He had a mission to improve his (and Navy’s) leadership skills to improve the performance of his ship.
This book had a lots of interesting insights on leadership and organizations that is relevant also outside US Navy context → warmly recommended.

The Chapters

A quote from the introduction of the book:

The book narrates episodes in Benfold’s two-year voyage thorugh uncharted waters of leadership, and is organized around the lessons I learned. A chapter is given to each one.

Mostly for my own sake, here’s a list of the chapters of the book:

Lead by Example

• It’s funny how often the problem is you.
• Never forget your effect on people.
• Leaders know how to be held accountable.
• Never fail the “Washington Post” test.
• Obey even when you disagree.

Listen Aggressively

• See the ship through the crew’s eyes.
• Find round people for round holes.
• Use the power of word magic.

Communicate Purpose and Meaning

• Make your crew think “We can do anything.”
• Open up the clogged channels
• After creating a gread brand, defend it.
• Freedom creates discipline.

Create a Climate of Trust

• Never pit dog against dog.
• Even the worst screwup may be redeemable.
• Welcome the bad-news messenger.
• Protect your people from lunatic bosses.
• Being the best carries responsibility.
• Trust also makes money.

Look for Results, not Salutes

• Help knock down the barriers.
• Let your crew feel free to speak up.
• Free your crew from top-down-itis.
• Nurture the freedom to fail.
• Innovation know no rank.
• Challenge your crew beyond its reach.

Take Calculated Risks

• Bet on the people who think for themselves.
• Take a chance on a promising sailor.
• If a rule doesn’t make sense, break it.
• If a rule does make sense, break it carefully.

Go Beyond Standard Procedure

• Keep your priorities in focus.
• Stay ahead of the competition.
• Push the envelope for innovation.
• Volunteering benefits everyone.
• Go for the obvious. It’s probably a winner.
• Don’t work harder. Work smarter.
• Don’t fight stupidity. Use it.

Build Up Your People

• Little things make big successes.
• Trust people. They usually prove you’re right.
• Newbies are important. Treat them well.
• Be the rising tide that lifts all boats.
• Build up your bosses.
• Expect the best from your crew. You will get it.
• Build a strong, deep bench.
• Counsel continuously – and honestly.

Generate Unity

• Forget diversity. Train for unity.
• Deal out punishment strictly but fairly.
• What’s bad for women is bad for your ship.

Improve Your People’s Quality of Life

• Fun with your friends makes a happy ship.
• The first priority: good food.
• Add to your crew’s bottom line.
• In heavy times, lighten up.
• Let the crew show off the ship.
• The secret of good work? Good play.

Some notes on internetworking (with a Mac)

Some time ago, I ended up repeating the basics of TCP/IP internetworking with a book from my bookshelf that I had bought from university library removal sale years ago, Douglas Corner’s Internetworking With TCP/IP Volume 1: Principles Protocols, and Architecture. I was a bit surprised to see that even though my copy is 3rd edition printed in 1995, a good part of the content was still pretty valid (routing protocols, for example, have change pretty much and HTTP was so young at that time that I don’t remember it being even mentioned).

After that I also found Ilya Grigorik’s book High Performance Browser Networking that can be read online for free. This book has a good summary of TCP/IP networking from the performance point of view.

This posting is a collection of notes from reading these books, mostly for myself. In addition, I’ve collected info on some basic networking tools, mostly on Mac.

Internetworking in general and Internet Protocol (IP)

With everything related to networking, protocols etc, it is good to be aware of the layer model of communication, check for example OSI Model.

IP in general

In the end, data is transferred in various physical networks that such as Ethernet, WLAN etc. Internetworking and Internet can be seen as creating an abstract network over these physical networks.

Internet Protocol (IP) defines the basic building blocks for this:

• Connectionless, unreliable, best-effort packet delivery system
• Delivers packets, also referred to as datagrams
  • A packet has header and data parts
• Addressing scheme, addresses for each interface
  • IPv4 32 bits (4 bytes), usually represented each byte as a number separated with dots, e.g. 123.124.56.3
  • IPv6 128 bits (16 bytes), usually represented as 8 groups of 16 bits, each groups as 4 hexadecimal digits, e.g. 2001:0db8:0000:0000:0000:ff00:0042:8329 (some abbreviation rules exist)
• IP address is divided to a network part & a host identifier within the network
  • Originally, IP address had fixed network part & fixed host part (only 256 networks)
  • After that, IP addresses were divided to classes of networks (different classes had different network-host division)
    • In early 1990’s, class system was replaced by Classless Inter-Domain Routing (CIDR).
  • In IPv6, the scheme is roughly: 64 bits for net and 64 bits for host.
• Routers/gateways connect physical networks to other networks. (These hosts are connected to multiple networks at a time)
• 3 ranges of IPv4 addresses reserved for private networks (can be connected to public Internet with Network Address Translation (NAT)
  • 10.0.0.0–10.255.255.255 (24-bit block)
  • 172.16.0.0–172.31.255.255 (20-bit block)
  • 192.168.0.0–192.168.255.255 (16-bit block)
• Each network interface of a host has a separate IP address
  • If a host is in two IP networks, it has a separate IP address for both networks

Some related protocols & commands

Internet Control Message Protocol (ICMP)

ICMP is a part of IP suite that delivers control information, for example:

• Destination unreachable
• Time exceeded (IP packets have time-to-live, TTL, in practice number of hops)
• Echo & reply (ping)

Ping (ICMP echo & reply):

ping

Trace route (with TTL)

traceroute 216.81.59.173

Address Resolution Protocol (ARP): Mapping IP address to a physical address:

Display current ARP entries

arp -a

Dynamic Host Configuration Protocol (DHCP)

DHCP is a protocol for dividing configuration (for example IP addresses) for hosts

• Client broadcasts request (to servers)
• IP address, router address, …

With Mac, one can check the DHCP packet content with

ipconfig getpacket en0
ipconfig getv6packet en0

Mac network interfaces

Network interfaces

ifconfig

Interface naming with Mac (BSD history) : http://superuser.com/questions/267660/mac-os-x-please-explain-ifconfig-output

• en0 : Ethernet 0
• en1 : normally AirPort (WiFi)
• lo0 : Loopback

Other related things

Print routing table (numeric network addresses):

netstat -nr

User Datagram Protocol (UDP)

UDP is a simple protocol built on top of IP

• Unreliable & connectionless delivery service over IP
• Adds source & destination ports (to distinguish destinations within a single host)
• Can be examined by the things handled by TCP that aren’t handled by UDP
  • No guarantee of message delivery or message order
  • Stateless
  • No congestion control
• For example WebRTC runs on top of UDP

Transmission Control Protocol (TCP)

• General-purpose reliable stream delivery service build on top of IP
  • Stream (ordering of TCP packages)
  • Connection & state
  • Buffering & congestion control
  • Full-duplex
• Adds source & destination ports
• Connection & state
  • Connection is determined by pair of endpoints (host, port)
  • TCP uses three-way handshake to establish a connection:
    • A→B SYN seq=x
    • B→A SYN seq=y, ACK x+1
    • A→B ACK y+1
  • See also simplified TCP state machine at Wikipedia
• Controls
  • Receive window (rwnd)
  • Congestion window (cwnd)
  • Details at Grigorik’s book, for example

Related tools

TCP connections can be examined with netstat (does not include processes with Mac)

$ netstat
$ netstat -p tcp

With Mac, connections can be listed with processes with lsof.

lsof -n -i4TCP
lsof -n -i4TCP:80 | grep LISTEN

Another command-line tool for displaying network information is nettop

Misc. tools

Mac firewall

http://support.apple.com/kb/ht1810

sudo ipfw list

Netcat

Netcat (nc) is kind of swiss-army knife for TCP/IP. Unfortunately Mac nc is a bit restricted compared to Unix/Linux versions.

Some links with examples (mostly Unix/Linux)

• http://en.wikipedia.org/wiki/Netcat
• http://www.g-loaded.eu/2006/11/06/netcat-a-couple-of-useful-examples/
• https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/nc.1.html

Others

To configure network settings from the command line, one can use networksetup

List of related topics / links

• DNS
• Routing
• HTTP
  • In the series of ‘old books with surprisingly relevant content’, see the first 3 chapters of Web Client Programming with Perl for a brief tutorial to HTTP.
• WebSocket
• Email (IMAP etc)

What is the end of the line (a couple of notes on newlines)

Handling newlines

When working with text files between different systems, line changes (also known as newlines) can be problematic. This is because different systems use different character codes (or combinations) to present newlines.

The basic codes used are the following:

LF (Line feed)	‘\n’	^J	0×0A	10 (decimal)
CR (Carriage return)	‘\r’	^M	0×0D	13 (decimal)

The basic cases for the typical systems are:

LF	Unix-like systems (including GNU/Linux & OS X, for example)
CR+LF	Windows & DOS, textual Internet protocols typically (see below)
CR	Mac OS before OS X

How about the (textual) Internet protocols? In general they use CR+LF on the protocol level, even though usually recommend applications to accept also plain LF. In textual mode, FTP transforms the newlines between CR+LF and system’s encoding (but not when in binary mode).

How to convert them

A typical example case is having a file in Unix/Linux/OS X that has Windows newlines. In this case, an extra ^M (CR) is found at the end of each line.

There are many ways to convert files between different newline formats (see Wikipedia). One utility for this for is flip (note to be a bit careful, since the tool updates the file straight)

Current file type can be determined with

$ flip -t file

File can be converted with -u (to Unix) or -d (to DOS/Windows), like

flip -u file_with_windows_newlines

Also tr can be used, like the following:

tr '\r' '\n' < macfile.txt > unixfile.txt
tr '\n' '\r' < unixfile.txt > macfile.txt

Links

• Newline article in Wikipedia
• EOL story
• ASCII control code chart (Control-M ^M & Control-J ^J)