Attila's Den

Just a quickie, Brett Terpstra (@ttscoff), the author of the sensational fork of Notational Velocity NVAlt has been active these past two days, and has put together an excellent (or, let’s say, Awesome, really it is) HTML to Markdown converter. This is still in the experimental stage, but will certainly delight all Edito users.

The tool is located here: FuckYeahMarkdown.com or at the more “politically” correct HeckYesMarkdown.com. I’m really longing to see where this goes, the first shots are really worth a look.

So, Edito has now gone through a few reviews (version 1.2 just accepted) and I thought it may be worth to share what I’ve learnt in the process of earlier submissions. See, I’m coming from an altogether different environment, which also may be your case. And it makes a big difference, particularly in the way we see a 1.0.

The first thing you may have read on the web is that Apple does not take half-finished products, eg beta or <1.0 versions of apps on the store. This is true, and will invariably lead your app to rejection. Now, 1.0 is a different beast when it comes to app submission than what you have experienced before. The reason is easy to grasp: an app published to the appstore has been QA’ed by Apple, and they require perfection in order to not alter the experience of the device - in my case the iPad. See, when you hid the iPad and your app crashes, who are you most likely to blame (got to hell… [fill the blanks]). My guess is either Apple or Steve in the first place, and then maybe you’ll think about the programmer of that app. With that in mind, your app needs to be perfect.

the usual process

If you have been developing for other environments, you will be familiar with going through alpha, beta, code freeze and release phases. If this was done well, a test plan should have been introduced well before or right after the alpha phase for QA. At this point, I can’t stress enough the importance of a test plan. We’ve got a pretty tedious one for the NIL Simulation Suite, that takes roughly a day to complete as most of it cannot be automatized at the moment. Yet, it provides a certain confidence that the product fits the bill, and you’re not going to be bothered by too many returns. Ideally, your bug list should be nearing zero by code freeze and release phases. That’s the engineering point of view anyway…

App submission: tougher, meaner, sharper

Lets make things clear: we’ve heard so much about app submission, that it’s not fair, that policies are not readily available, that we cannot understand why some apps are rejected, that Apple is dismissing some apps because it would compete with their products, etc… This is all wrong. Objectively, I just failed the software review twice because I was not aligned with what was expected from an app, and it was my own mistake. Nothing to do with them. Here’s my 2 cents about the process.

1. Make sure you are bug-free
   This is first and foremost requirement when your app is examined. While you can certainly accommodate or “hide” some bugs in a regular software - I don’t know one that is nit bugged in a way or another - your app should be completely bug free. A few hints to do so. First treat every warning of the compiler as a potential bug. The xcode compiler is excellent, and goes to a large extent beyond most other compilers I’ve used. The debugger, however, is not as good as Eclipse, but with a bit of practice, we can make do. There are some precious tools as well, such as the “build and analyze”, which tests the logic of your app after building it. Definitely a must-use to test your pointers, in particular their initialization as well as the memory state. I should put together another post for the best practices of debugging in Xcode, as a string of ore tools are quite useful in later stages.
2. Everybody says so, it must be true: comply with the HIG.
   I’m not sure whether the human-interface guidelines are here to help or to bother, but they’re here. The first rejection of Edito was due to the simple fact that we set the interface as portrait-only, and I did not think about including the 180 degrees rotation, confident that the user would have the iPad in it’s regular position, button down. Completely wrong. You cannot argue your case with a reviewer, who remains an anonymous tester in Cupertino. Or somewhere else, doesn’t matter. So, beware of orientation, of providing a unified interface - don’t pick an icon from a collection and one from another, all this sort of things.
3. Establish a complete test plan, stick to it.
   Again, what should be standard practice for a regular software is a must for an app for iPad. Basically, figure this out: the tester is going to try all possible combination of strikes to test if your software responds to all events without crashing or breaking the HIG, and if it does what it is supposed to do. Save yourself time and effort, and do it before him/her. The second review failed by Edito was just such a case, where some popovers did not dismiss when a certain sequence if keys was used. While I do not think that anyone in his rightful mind would use that sequence of events, the rejection was right and well motivated. Or at least I could understand it. Notice as well that the testers were nice enough to respond to an email requesting clarifications on hhe test results: they did re-test he bug and send 3 additional reports, which they were definitely not obliged to do. Now the action so that kind of inconvenience does not show up on your front door, is simple. Just establish a test plan that accounts for all possible orientations, change of orientation, and combination of touches (note: if it’s a design/drawing app, it’s not going to be possible to test all touch events, but all sequences of touch events should still be tested for). This test plan will be re-usable if you record it in Xcode analyzer, which I would also recommend.
4. Use the tools.
   The Xcode analyzer is your friend. It will check memory leaks for you while you are running the app in the simulator or on the Device. I do personally also check that the files input and output are neatly open & closed, and that everything behaves like it should. Memory leaks checking is very important. The main argument is that there is no garbage collection in objective-c, which is kinda true only. In fact, only those variables initialized using the alloc statement will need to be released, any other variable created using eg NSInteger or NSArray or NSWhatever (!) will be deallocated when we’re coming out of the block where it stands. For an order of magnitude, Edito runs all the time with less than 3Mb of RAM, which is to be expected for a text editor.
   One more word about the debugger: while we grow increasingly accustomed to integrated tools, I found that the good all ways are good enough for most tasks. In other word, using a simple NSLog statement of your variable is good enough in most cases, particularly if you’ve been debugging along the way.
5. Last check before Gunhill
   Think you got it right? Well, it’s the last time to enlist beta testers, who will run your app in a real situation. You’ll see that this forces you to polish some of the aspects that were maybe not so practical, but you learnt how to do with. Your testers are likely to pick them easily and send them back to you as bug or feature requests. Think about it twice before dismissing them uor delaying changes, these issues are likely to be the first ones to pop-up after you submitted the app. Remember that the customers of the appstore are exceedingly demanding when they ace paid for an app. This, it will need to be perfect, polished and easy to use and interface with. Remember as well that some of the HIGs are recommendations, and not causes to fail, basically. It means that your app may well move forward, but it will still be hampered in it’s execution by those small problems you have not fixed. And will suffer from bad reviews, which means you’re unlikely to sell afterwards. So, better loose 1 week to fix everything than commit a half-done version which will piss the users and gain bad reviews.

Closure

To summarize, not only your app will need to be perfect in Apple’s sense, but it will also need to be DICEE, in Guy Kawasaki’s terms. If you haven’t yet, go check his talks about innovation on YouTube - try this and look at related videos, “the art of the start”. You’ll learn that the app should be deep, intelligent, concrete etc. And to add a bit to it, just figure that if you had the best idea, tested your idea, developed your idea, you’re just one step from seeing it to fruition. That step is definitely boring, you’re in a hurry to get your 1.0 out, but you need to figure out that it will not get out if you do not put the last effort, last drop of sweat (and tears) into it!

We’ve just seen in the previous post Building Edito - Part 2 that we are following a 5-gated process to developing our app. By the way, the name popped-out later in the process, at the early stages I just called it iMarkdown. With most of the stages 1 & 2 well engaged, the big remaining task to get into stage 3 is to draw a list of features. Because Objective-C is object oriented, this list will likely turn into our main classes.
My method is the following:
1. get to a white board
2. start throwing ideas on it 3. second round: organize around one central idea
4. third round: time to introduce some logic into this mess. Which feature derives from which, essentially.

Now, this is the tricky part. When you’re done, make a clean representation of what you have, and distinguish between what is horizontal and vertical. Let me explain. Horizontal is everything that the user is going to interact with. Vertical is the “engine” of the software, ie. in our case the Markdown conversion, reading and writing to a specific format, the various libraries stored or called. In other words, the graph you will end up with is in two parts, where the first one (horizontal) gives a pretty good picture of the app UI, and the second (vertical) the built-in functionalities. The vertical part is easy here, we just want to convert ASCII to HTML following the Markdown syntax. The horizontal part, well, is depicted on the figure below.

On this pic we have everything we want and more. You can see that I’ve drawn an ellipsis on the features that we’re keeping for the version 1.0. That comes right after putting everything down on paper, and it’s the analysis of needs vs wants. On the graph, I’ve got all I want — and a potential customer may want or expect from such an app.
We could group everything as essentials and non-essentials, but that’s probably a bad calculation at this point. For instance, what would you do with an app that can just type, open and save documents? Hate it probably, or not use it more than once. What we’re set to achieve is an easy way to replace all the operations we’re doing in our favorite Markdown editor on our laptop or mainframe computer. We’re generally using it to produce HTML documents. Those generally contain links, images, and I’m constantly switching (at least I was, in the beginning) to a Markdown cheat sheet. All of that I want in my app. Then, I want to be able to send those documents. Let’s face it, iPad is not so cool to exchange documents, and iOs 4.2 is not solving that problem apparently (we would have been told otherwise). So, eMail gotta be a feature. In the end, I’m proposing to leave out (for now) what brings little (word count), complicates the operation (importing USER css) or just is impossible to support - I do not have time to spend checking the css files of users - and what I can’t implement in say 2-3 days. That includes syncing to other APIs, the spell check, and importing pictures.
This last one is a tricky one, as I’d like to be able to use pictures from my iPad library, and just include them naturally in the document. The best way to do it is probably to convert the picture to Base 64 and paste it in my document, but it’s going to look truly ugly, unless I find a way to make it magically disappear while being retained in the document. That’s actually going to influence our document format, this idea. Syncing to other systems was probably the toughest one to reject, as I know everyone wants these features. Yet, bear in mind I’ve got to learn Objective-C first, get to writing my own app, and get it accepted on the App Store, ie get through a full process loop, before I envisage to use some private APIs that may be tied to specific conditions…

All of that being done, I evaluate a ball figure of 2 months to get this thing up and running. I don’t have much more time, and since it’s not rocket science - I would do it in Java in a quarter to maybe half the time probably - this should be OK. The main difference with doing it in Java is that 1) I don’t know the language, although I now have a pretty good idea of what it looks like after browsing the first few chapters of reference books, and 2) I’ve got what appears to look like a solid, fully populated dev. environment.

Next part will be quick, it will just give references I used to learn Objective-C. Two weeks were good enough to grasp most of it…

Edit:

There are two important things I forgot, and that came afterwards. Well, in fact, 3.

1. there are some accessibility facilities in iOs now, so that a visually disabled could use the software. That comes at the price of a small effort, but since we’re aiming (and have achieved) at a clean, uncluttered interface, this should be possible. And give me a certain peace of mind;
2. producing PDFs… This does not look like an horizontal only task, and that’s why I dropped it originally, although I thought about it - and forgot about it. That’s the danger of splitting horizontal from vertical, you tend to forget some important things in the vertical domain to concentrate only on what the user will experience!
3. printing: this looked unreachable until iOs 4.2 was announced, but is not quite that out of reach now. I’ll definitely have a look into this for version 1.1 or 1.2, depending on the time it takes to implement and review the software.

We’ve just been through my initial thoughts about this app. A bit more planning is needed before we get into “real” business, ie. engineering stuff. I find it not only necessary, but really useful, to follow a process, even though I’m the only person concerned with this “project”. What I’m using to keep track of the progress is a 5-gated process, which derives from a manufacturing process I learnt years ago while interacting with Thomas Celluci, then COO of Zyvex, a Nanotech company. It is highly customizable and has proven to conform to most projects I carry out.

5 Gates to pass for a release

Here are the stages followed:

1. Stage One: Concept Feasibility and Evaluation

The goal of Stage One is to investigate new product opportunities. I start by looking at a particular application, research the scope of the product(s) that are required to meet the application, estimate the required resources, and then produce a Preliminary Product Plan. The first step of the Plan is to create a product description and a project overview to ensure that the product is aligned with our vision, core competencies, and the specific applications we’re targeting.

We define potential product attributes (features, benefits, reasons to buy), estimates market size in view of current market growth trends, and creates a preliminary product applications overview. We also put together a list of potential customers and competitors.

2. Stage Two: Planning and Specifications

The goal for this stage is to develop a complete set of marketing specifications (using the Preliminary Product Plan) and produce a Complete Product Plan (CPP). The CPP covers marketing opportunities, financial considerations, and the design concept. We also establish a “needs vs. wants” list, a development budget, source external resources, and a detailed schedule. We perform a SWOT-competitive analysis (strengths, weaknesses, opportunities, and threats), a scenario analysis, and a sales forecast estimate. We also check key IP disclosures if any.

Note: it is very important to make sure you are not going to use a private API at this stage. This will get you kicked out of the AppStore at submission.

3. Stage Three: Core Development

The goal for Stage Three is to complete design and development and to collect feedback on the product beta. We use the Complete Product Plan as the foundation for creating a design that meets the target specifications. We can worry only about dev at this stage, as the rest of the Marketing and Sales will be taken care of by the AppStore.

At this stage, we produce the core functionalities of the app. No programming ever took place before now, and it’s relieved that we, at last, get to open Xcode and peek at our development tasks. If need be, we revise the development schedule at this point.

4. Stage Four: Interface Development and Integration

The goal is to secure a high-confidence hand-off to the AppStore. I thought about splitting this stage in two, but in essence I’ve always kept it as is. The main reason is that I’m constantly switching between tasks at this point. I’ve got a working engine for the app, which means that it runs in the console. I now need to deal with the UI, and have found quite impractical to do so by considering the UI alone. Here are the tasks: * build an “ugly” UI, where all the elements are named, buttons are in place, and test all elements separately. I generally add elements one by one, together with their actions and link them to the core functions
* graphics: I realized years ago that I can’t draw, let alone create anything appealing to the eye. I’m lucky enough to have my own private graphics designer (Ivanna, my Wife), who ends up looking at the various pieces of design, and remains a critical assessor during the process
* build a logo when the app is finished. It’s easier to pick up some elements of your UI when your UI is here. * get it to work together… that’s the hard part! At stage four I concentrate only on getting the software running and performing as expected in the Simulator. There’s a general mop that comes up in next stage * prepare a webpage for the product, examples of use

Note: Now is the right time to read again Apple’s Human Interface Guidelines (HIG), if you want to make it past the review process (assuming your app performes as advertised).
Needless to say, I forgot to do so which wasted me one week at first submission for a really stupid item (interface rotation) - more about this in a coming post.

5. Final Stage: Build and Release

The goal of the final stage is to demonstrate product assembly according to QA standards, and deliver the product to the AppStore. This iterates between iTunesConnect and the Device. Basically, generate the signing certificates to transfer the software to your iPad, run tests and modify code until it runs like in the Simulator. Enlist some beta-testers and send them a copy, with extracts of the CPP to guide them in testing. That’s the final chance to change the product, and is of major importance! Last step, when everything’s good, is of course to submit to the AppStore.

This is what we do

Really, if you followed the first post (Part 1 below), we’ve almost completed Stages 1 and 2. We just need to put down a list of features and perform a needs vs. wants analysis. That’s key to getting into core development, and that’s what we’ll do in next Part.

So, we’re waiting for the release of Edito, our Markdown viewer/processor/integrator for iPad. We’ve had quite a large number of hits on the webpage we’ve put up, as well as a string of emails, so we’re hoping it will be released soon by Apple on the AppStore. I’m thinking about putting up a series of posts to tell you — the user — how Edito was developed, the whole genesis, and some of the pulleys and crowbars. I guess I’ll be missing some of the milestones I followed, but I’d rather try to put it on paper/screen while it’s still fresh in my mind. So, let’s get started. First thing first, who-what-why-whyme?

Existential questions

What

Let’s start with the “What” and “Why”. Pretty obvious, I was planning to buy an iPad, and looked for the apps that would best fit this new format. One of the things that I was interested in was a way to type documents on the go. In essence, I used to travel a lot, and my laptop, a MacBook Pro, although it gives me 4 good hours of battery life, was not enough for a transatlantic trip. Enters the iPad, with 10+ hours of battery life… in fact, way more, I happen to use it all day for work without recharging, I guess to get to 10 hours only you must run it for viewing movies or gaming - whatever. Anyway, among my needs, a good text processor comes top of the list. Looking over the app store, and googling “iPad Markdown” do not give any tangible result. Jumping to conclusion after a bit more research, there is no Markdown converter/viewer for iPad. Interesting?

Why

Why would that be? There are a string of possibilities, and I went through most of them I caught up 3 that were the most tangible:
1. it’s not easy to implement, so developers who have thought about it have either given up or not succeeded, or considered that the result was not worth the effort
2. you can’t use a script to carry out the conversion, so it’s one of those cases where we need to re-invent the wheel
3. there is no commercial interest for this kind of app…

Let’s start with number 3: I picked up on the web a nice review with stats about the spread between various types of apps in the App Store. Comes out that only 10-12% of app are filed as “Productivity” tools, while more than 80% of the Store is about gaming. Well, given the growth rate of the Store, and the number of devices sold every single month, there is a good potential user base. Even if you’re touching only 1% or 0.1% of the whole park of devices out there, that’s still a huge market.
Number 2, I did not think about straight, it came a bit later, when I was researching about Objective-C and iPhone/iPad programming. Comes out that it is impossible to use a perl script inside an app. On a Mac, not a problem, you define a NSAction and bang you’re done, but not on an iDevice. That’s a bad blow, because it means that you need to program an in-app converter, in C or Objective-C ideally. See, if this was not the case, I could have simply used John Gruber’s Perl script and be done in a breeze… but so would have other devs. Number 1, that’s a worry only when you get into the programming phase, and sweat over yet another freaking parser. Did I mention I hate parsers? Every developer of the planet must do too… Anyway, let’s get to the who-whyme and not be too afraid from all of that.

Who-WhyMe

My main problem here is not to develop or not to develop, it’s to evaluate if this is possible, and whether I’ll have time to go through it. I’m not a developer per se, by which I mean tha tI’m not a computer engineer. I’ve been trained as a Materials Scientist, and took a PhD in the same branch, in polymer physics. I’ve honestly loved it, and still do. Yet, I’ve got my own Company to run, and we’re involved in such a small niche that our distribution is not easy, although developing for these Nanotech processes is really interesting.
Anyway, I’m well trained in a dozen of languages, and since I’m away from the labs now I spend most of my time developing. My two languages of choice are Fortran (I think in Fortran) and Java (I speak in Java), and I mainly use a simple text editor for Fortran programming and Eclipse for Java. All the NIL Simulation Suite is programmed in Java by the way. Like any good scientist, I’ve also had to deal with C, Python and C++, although C++ has always confused me a bit.
Being a scientist can take various meanings, but for me it covers a few essential qualities: you must be patient, and very resilient. As I told my future boss when interviewing for my PhD position: when I bite a bone I don’t let it go. And most of all, you must not be discouraged easily, otherwise Research is going to be very hard for you, as you quickly learn that things are never over.
Armed with this, I can probably learn a new language, Objective-C, in two weeks to a month, and try to develop a parser for Markdown to HTML conversion. Cranky, but no big deal. And it comes at a good time in my personal/professional life: wife and kid on vacation, and I’m definitely getting tired of programming polymer physics, been on that nano-imprint problem for two years day and night, I need a break. It’s either that or programming a 3D Tetris or Rubik’s cube. Got to be honest though, I also researched the possibility to get a 3D visualization engine for Google Sketchup and other CAD formats up and running for this platform, but I’ve been having so many problems with converters/parsers of 3D geometries lately for the NSS that I decided to pause this. Really needed something else to distract me.

Conclusion

For me it’s a go. Buy a few books and read the intro chapters to see where we’re going in terms of technology and implementation. Get real, do a first plan of development and jolt down a few sketches to serve as a guide during the reading/assimilation and maybe development phase. Point is, I’ll be reading and learning selectively the advanced features of the language and the interface once I’m done with the basics of Objective-C. I’ll complete a few strategic points before going further, I’m still way way far from writing my first line of code!

Strategic inputs

As I said, I had a look for 1-2 hours at the business case, and it looked OK, although I could not pin some precise figures on how much the app would make. But it seems that selling such an app between 5 and $9 has potential. I did a quick SWOT analysis as well, which in general helps me to focus on the essential afterwards. I left all the fuzzy questions such as the definition of a mission statement (I don’t give a shit, that’s Corporate bull) and the implications of not achieving this strategy: I’m not trying to convince anyone but myself, and keep a few notes while I’m going through the creative process. So, simple SWOT with 2 bullets under each category:
1. Strengths: Markdown is really easy to use, and would make a perfect replacement for a rich-text word processor on the iPad. Dealing with text only files (ASCII) makes sense because of the very low memory and storage footprint envisaged;
2. Weaknesses: typing on the iPad, come on!? That’s a reduced keyboard, I can’t find the characters I need for Markdown in less than 3 strokes. Remember how Apple revolutionized the Music industry with the iPod? every single function was available in less than 2 strikes, which I still admire: the device was cool, but the software was a masterpiece in terms of usability. There are a few other weaknesses that could be mentioned, but I went for the fact that when I’m typing text in Markdown, I’m constantly switching between the app and the web-browser to gather links and pictures… which is not possible on the iPad. Well, it’s possible, but it means going out of the app and coming back after having copied a link, it’s not handy at all. And although the app’s got nothing to do with it, it’s still going to piss me off.
3. Opportunities: there’s no other app that is doing Markdown to HTML conversion out there! Tumblr is, in my mind, the best and easiest blog engine out there, and it takes Markdown text as an input…
4. Threats: any guys out there developing the same? Note to myself: keep Googling this entry along the process, check blog entries and discussions to identify key players. There are already a good number of text processors, they may come up with an innovative idea on how to do RTF in a way that complies with the device. Check what the (much admired) Omnigroup is doing, what Marco.org is up to, MarsEdit, the guys who do Coda and CSSEdit… basically keep up-to-date with all the big Guns: they’ve got the experience, the name and respect in the industry and among users - and they well deserve it. With 15 years of expertise in another field, you’re a newborn to this industry, don’t know the codes and rules, better be humble - another fresh start.

That’s all folks

Good way to start, we’re going to maximize our strengths, which are really those of the Markdown language, try minimize our weaknesses, achieve the opportunities and monitor the threats. Worse come to worse, someone else may come up with the same idea and do it better and faster than I, but I will at least have learnt a new language and system, and will be able to use it for something else (got other ideas relating to Engineering - exciting!).

That’s all for now, in the next post I’ll be scribbling on the blackboard my features requirement, which will lead to the development plan. What’s happening in between? Read everything you can, get to know what’s out there, read products reviews, look at interfaces, and get to understand the terms you’re not familiar with in the first chapters of those iPhone dev books.