On Accessibility in iOs Development, Implementation in TipToe
It’s a wide world
Let’s start with the obvious: it’s a World Wide Web, and we tend to forget about it while being entrenched in one way or another in our lives and work activities. It sounds like I’m going to be apologetic, which is not frequent, and I am. I’ve recently developed and published through the AppStore an app called TipToe, see it here and follow the link to the store — $1, cheap, indispensable (it sure is ;-). So, that’s a cute app that lets you search Bing and Blekko, and aggregates the search results in a nice table. And I’ve put in a minimal in-app browser (minimal, yet complete) which is lean and fast.
The app has been featured on [MinimalMac](http://minimalmac.com], we proudly sponsored the RSS feed the week of the release of TipToe (entry on the 4th of April), and it was picked recently by LifeHacker. Life couldn’t be better (well, actually, we could always sell more, but that’s another story). Couldn’t? Sure it could, and should.
A few hours after we got featured on LifeHacker, I received a moving email from a user who purchased the app, regarding the lack of accessibility of TipToe. This user is visually impaired, and complained that VoiceOver in TipToe was inexistent… Well, actually, being a really sweet person, he sent an email noticing that some of the buttons were not customized for VoiceOver access. Indeed, that’s one of those emails that hit you pretty strong in the face, and for two reasons. One, I remember reading a very good blog post by someone visually impaired explaining how the iPhone gave him back a sense of sight by letting him know which colors were in front of him. This post moved me (note to self: gotta find it in my archives). And two, there was an excellent post from Matt Gemmel here that describes how easy and important it is to make your app accessible. So, shame on me for forgetting about it, I read that piece, enjoyed it and bookmarked it for using it later. Only problem is I skipped using it. Now, pretty much like anything I do, I just considered that shortcoming as an engineering problem, ie. a mistake that should be fixed. Off we go, let’s add Accessibility to TipToe. Matt having indicated that it’s easy, I shouldn’t have many problems to implement that. Quite wrong, but here’s what I did in the end.
The elements to be made accessible
I’ve made a quick capture of some screens from TipToe, and here we’re going to address two kinds of tips: 1- making the table cells more readable, and 2- accessing some elements that do not have their Accessibility property reachable from Interface Builder (IB). The latter is probably the most interesting case. In TipToe, I had to trade the Navigation bar at the top of the view for a standard toolbar. Basically, I wanted to define an empty button in the center of the toolbar so that I can make the web address bar disappear on touch. That’s a neat trick to maximize the screen earl estate, and it saves me from a scroll view, which I don’t like when browsing (always need to scroll back up to get the actions, and I wanted the actions button to be always visible, as they open the share dialog). There’s certainly other ways to achieve that, but it was the easiest and quickest to implement. You’ll notice as well that I haven’t circled all the elements on the left hand side of the picture. Basically, the UITextField for search entry and the information button at the bottom right are already addressable in IB, so no need to do a fuss. This case is well documented in both Apple’s Guidelines and Matt Gemmel’s post.
There’s a good hint in Apple’s Accessibility Guidelines, which I shamelessly use here. It consists in concatenating some fields displayed in your cell in order to make it more easy for the robot to read.
Here is the snippet for the table:
So, quite simply, I’ve got a custom cell which displays the title (re-formatted to make it homogeneous in my app, the url (ditto) and the abstracted content (that’s the description label). This is done in a most classic way using three UILabel (Lines 1-3), for which I’m assigning the .text property after, while extracting the search results from the main table of the class (Lines 4-6). Now, the catch is that VoiceOver reads everything in a cell when it is tapped. And clearly, you do not want the url to be read, it will literally read “h t t p column slash slash etc…”. Not really pleasant. Instead, we’re just merging the titleLabel.text and descriptionLabel.text, and assigning the result to the cell.accessibilityLabel (Line 8). This is now the text that will be read when the cell is tapped. That was pretty easy, let’s get to the hairy section now.
I’m getting away in the Display view with only changing the viewWillAppear method, such that:
Now, there’s plenty to discuss here. The main point is that all those elements do not have their accessibility property accessible from Interface Builder, so we’ve got to be a bit smart. We’re going to use the same trick for all the elements, which consists in accessing the accessibility properties of the view that contains the element.
First up: the top bar. That’s a UIToolbar, I’m accessing it through a tag. Nothing much to say about this… except that we need that UIToolbar to be defined for the next step.
Second: let’s get into the bar buttons. Once you know how to do it, it’s evident. We’re just picking up the UIView that contains the button (Lines 7 & 10), and doing so from the UIToolbar list of items. These items are numbered in the order they are displayed in the IB, starting with 0. In my view I’ve got a space before the first button so my button1 is at index 1 (Line 7). The all shebang is then over, as I can set the accessibility label and hint of my newly defined UIView (Lines 8-9 and 11-12). Simple enough! Unfortunately the UIBarButtonItem accessibility cannot be accessed directly from IB, but this does the trick in a few lines.
Third: well we’re got this damned UISegmentedControl. Let’s face it, life would have been much easier if I had kept the first version of the app (alpha), which had two separate buttons. But no, I had to get it with the look and feel of Apple’s Mail and Safari, which use this UISegmentedControl. Now, the same trick is used, with a twist. We start by picking up the UISegmentedControl (Line 14), and then we’ve got to navigate through its subviews (Lines 15-29). We’re doing so using a for loop (Lines 16-29) over the [segmentControl subviews], and together with a simple switch statement (Line 18) we get our subview accessible. In our case, we’ve just got two segments, but this could be used for more. That’s all there is to it…
That was a quick rundown, and hopefully this post will be obsolete in a future release of iOS. Quite unfortunately, the doc on this subject is very light, so I hope it may help someone. The take-home message is simple:
- if the element has its accessibility property available in IB, just use that;
- if it does not, I’ve either covered the case here, or you’ll have to dig through your elements inheritance and find the view for which you can set-up the accessibility.
Questions and comments are always welcome, though I’m afraid I don’t have much time to check your code these days… So, please don’t hold it against me if I can’t respond fast enough.
NSRegularExpression & Brackets
I’ve been looking recently at a decent and elegant way to replace portions of a string of characters. While searching and replacing parts of regular strings, for example “foo” by “bar” is a built-in functions of iOs since the beginning, doing the same for “[^whateverfoobar]” in a sentence is clearly a pain, as one’s got to find the first two characters, the last one, then hold a copy of “whateverfoobar” and perform operations on it. So, is there a good way to find custom delimiters? Answer is YES since iOs4.0, where regular expressions have been implemented. Up to that point, devs were using a RegExKit, which is no longer needed, and may indeed cause problems when submitting the app to the AppStore. But anyway, one can do without now.
So, Apple introduced in iOs4.0 a new class, NSRegularExpression, which is exactly what we need to deal with such “[^…]” expressions easily.
I do not want to get into the syntax of regex, but a few points must be mentioned before looking at our example:
1. iOs follows the ICU convention, which can be found here;
Let’s look at that regex now. It looks like this
where the first character, a bracket “[” needs to be escaped by “", ditto the second “^”, follows the (very) cryptic “.+” that means any set of characters but at least 1, and the closing bracket “]” does not need to be escaped. That’s the standard, and we can check it on Zytrax, which results in:
Pretty cool uh? This little script gives us the expressions found matching the regex, enclosing the first one with
<>, and then gives the other occurrences in the string. With the small example we chose we can check that double brackets without the caret “[foo]” are not taken, nor is the empty “[^]” chain of characters.
Now, from what I’ve said before, in Objective-C, we need to escape characters a bit differently, such that:
If we do not do so, we’ll get get a Warning “Unknown escape sequence ‘\^’” which bugged me for a while (hence this post).
Here’s a small bit of obj-C code that demonstrates the use of our newfound toy:
And that’s it for now… Note to myself: I need to set the code display properly here…
Building Edito - Part 3
We’ve just seen in the previous post Building Edito - Part 2 that we are following a 5-gated process to developing our app. By the way, the name popped-out later in the process, at the early stages I just called it iMarkdown. With most of the stages 1 & 2 well engaged, the big remaining task to get into stage 3 is to draw a list of features. Because Objective-C is object oriented, this list will likely turn into our main classes.
My method is the following:
1. get to a white board
2. start throwing ideas on it 3. second round: organize around one central idea
4. third round: time to introduce some logic into this mess. Which feature derives from which, essentially.
Now, this is the tricky part. When you’re done, make a clean representation of what you have, and distinguish between what is horizontal and vertical. Let me explain. Horizontal is everything that the user is going to interact with. Vertical is the “engine” of the software, ie. in our case the Markdown conversion, reading and writing to a specific format, the various libraries stored or called. In other words, the graph you will end up with is in two parts, where the first one (horizontal) gives a pretty good picture of the app UI, and the second (vertical) the built-in functionalities. The vertical part is easy here, we just want to convert ASCII to HTML following the Markdown syntax. The horizontal part, well, is depicted on the figure below.
On this pic we have everything we want and more. You can see that I’ve drawn an ellipsis on the features that we’re keeping for the version 1.0. That comes right after putting everything down on paper, and it’s the analysis of needs vs wants. On the graph, I’ve got all I want — and a potential customer may want or expect from such an app.
We could group everything as essentials and non-essentials, but that’s probably a bad calculation at this point. For instance, what would you do with an app that can just type, open and save documents? Hate it probably, or not use it more than once. What we’re set to achieve is an easy way to replace all the operations we’re doing in our favorite Markdown editor on our laptop or mainframe computer. We’re generally using it to produce HTML documents. Those generally contain links, images, and I’m constantly switching (at least I was, in the beginning) to a Markdown cheat sheet. All of that I want in my app. Then, I want to be able to send those documents. Let’s face it, iPad is not so cool to exchange documents, and iOs 4.2 is not solving that problem apparently (we would have been told otherwise). So, eMail gotta be a feature. In the end, I’m proposing to leave out (for now) what brings little (word count), complicates the operation (importing USER css) or just is impossible to support - I do not have time to spend checking the css files of users - and what I can’t implement in say 2-3 days. That includes syncing to other APIs, the spell check, and importing pictures.
This last one is a tricky one, as I’d like to be able to use pictures from my iPad library, and just include them naturally in the document. The best way to do it is probably to convert the picture to Base 64 and paste it in my document, but it’s going to look truly ugly, unless I find a way to make it magically disappear while being retained in the document. That’s actually going to influence our document format, this idea. Syncing to other systems was probably the toughest one to reject, as I know everyone wants these features. Yet, bear in mind I’ve got to learn Objective-C first, get to writing my own app, and get it accepted on the App Store, ie get through a full process loop, before I envisage to use some private APIs that may be tied to specific conditions…
All of that being done, I evaluate a ball figure of 2 months to get this thing up and running. I don’t have much more time, and since it’s not rocket science - I would do it in Java in a quarter to maybe half the time probably - this should be OK. The main difference with doing it in Java is that 1) I don’t know the language, although I now have a pretty good idea of what it looks like after browsing the first few chapters of reference books, and 2) I’ve got what appears to look like a solid, fully populated dev. environment.
Next part will be quick, it will just give references I used to learn Objective-C. Two weeks were good enough to grasp most of it…
There are two important things I forgot, and that came afterwards. Well, in fact, 3.
- there are some accessibility facilities in iOs now, so that a visually disabled could use the software. That comes at the price of a small effort, but since we’re aiming (and have achieved) at a clean, uncluttered interface, this should be possible. And give me a certain peace of mind;
- producing PDFs… This does not look like an horizontal only task, and that’s why I dropped it originally, although I thought about it - and forgot about it. That’s the danger of splitting horizontal from vertical, you tend to forget some important things in the vertical domain to concentrate only on what the user will experience!
- printing: this looked unreachable until iOs 4.2 was announced, but is not quite that out of reach now. I’ll definitely have a look into this for version 1.1 or 1.2, depending on the time it takes to implement and review the software.
Building Edito - Part 2
We’ve just been through my initial thoughts about this app. A bit more planning is needed before we get into “real” business, ie. engineering stuff. I find it not only necessary, but really useful, to follow a process, even though I’m the only person concerned with this “project”. What I’m using to keep track of the progress is a 5-gated process, which derives from a manufacturing process I learnt years ago while interacting with Thomas Celluci, then COO of Zyvex, a Nanotech company. It is highly customizable and has proven to conform to most projects I carry out.
5 Gates to pass for a release
Here are the stages followed:
1. Stage One: Concept Feasibility and Evaluation
The goal of Stage One is to investigate new product opportunities. I start by looking at a particular application, research the scope of the product(s) that are required to meet the application, estimate the required resources, and then produce a Preliminary Product Plan. The first step of the Plan is to create a product description and a project overview to ensure that the product is aligned with our vision, core competencies, and the specific applications we’re targeting.
We define potential product attributes (features, benefits, reasons to buy), estimates market size in view of current market growth trends, and creates a preliminary product applications overview. We also put together a list of potential customers and competitors.
2. Stage Two: Planning and Specifications
The goal for this stage is to develop a complete set of marketing specifications (using the Preliminary Product Plan) and produce a Complete Product Plan (CPP). The CPP covers marketing opportunities, financial considerations, and the design concept. We also establish a “needs vs. wants” list, a development budget, source external resources, and a detailed schedule. We perform a SWOT-competitive analysis (strengths, weaknesses, opportunities, and threats), a scenario analysis, and a sales forecast estimate. We also check key IP disclosures if any.
Note: it is very important to make sure you are not going to use a private API at this stage. This will get you kicked out of the AppStore at submission.
3. Stage Three: Core Development
The goal for Stage Three is to complete design and development and to collect feedback on the product beta. We use the Complete Product Plan as the foundation for creating a design that meets the target specifications. We can worry only about dev at this stage, as the rest of the Marketing and Sales will be taken care of by the AppStore.
At this stage, we produce the core functionalities of the app. No programming ever took place before now, and it’s relieved that we, at last, get to open Xcode and peek at our development tasks. If need be, we revise the development schedule at this point.
4. Stage Four: Interface Development and Integration
The goal is to secure a high-confidence hand-off to the AppStore. I thought about splitting this stage in two, but in essence I’ve always kept it as is. The main reason is that I’m constantly switching between tasks at this point. I’ve got a working engine for the app, which means that it runs in the console. I now need to deal with the UI, and have found quite impractical to do so by considering the UI alone. Here are the tasks:
* build an “ugly” UI, where all the elements are named, buttons are in place, and test all elements separately. I generally add elements one by one, together with their actions and link them to the core functions
* graphics: I realized years ago that I can’t draw, let alone create anything appealing to the eye. I’m lucky enough to have my own private graphics designer (Ivanna, my Wife), who ends up looking at the various pieces of design, and remains a critical assessor during the process
* build a logo when the app is finished. It’s easier to pick up some elements of your UI when your UI is here. * get it to work together… that’s the hard part! At stage four I concentrate only on getting the software running and performing as expected in the Simulator. There’s a general mop that comes up in next stage * prepare a webpage for the product, examples of use
Note: Now is the right time to read again Apple’s Human Interface Guidelines (HIG), if you want to make it past the review process (assuming your app performes as advertised).
Needless to say, I forgot to do so which wasted me one week at first submission for a really stupid item (interface rotation) - more about this in a coming post.
5. Final Stage: Build and Release
The goal of the final stage is to demonstrate product assembly according to QA standards, and deliver the product to the AppStore. This iterates between iTunesConnect and the Device. Basically, generate the signing certificates to transfer the software to your iPad, run tests and modify code until it runs like in the Simulator. Enlist some beta-testers and send them a copy, with extracts of the CPP to guide them in testing. That’s the final chance to change the product, and is of major importance! Last step, when everything’s good, is of course to submit to the AppStore.
This is what we do
Really, if you followed the first post (Part 1 below), we’ve almost completed Stages 1 and 2. We just need to put down a list of features and perform a needs vs. wants analysis. That’s key to getting into core development, and that’s what we’ll do in next Part.
Building Edito - Part 1
So, we’re waiting for the release of Edito, our Markdown viewer/processor/integrator for iPad. We’ve had quite a large number of hits on the webpage we’ve put up, as well as a string of emails, so we’re hoping it will be released soon by Apple on the AppStore. I’m thinking about putting up a series of posts to tell you — the user — how Edito was developed, the whole genesis, and some of the pulleys and crowbars. I guess I’ll be missing some of the milestones I followed, but I’d rather try to put it on paper/screen while it’s still fresh in my mind. So, let’s get started. First thing first, who-what-why-whyme?
Let’s start with the “What” and “Why”. Pretty obvious, I was planning to buy an iPad, and looked for the apps that would best fit this new format. One of the things that I was interested in was a way to type documents on the go. In essence, I used to travel a lot, and my laptop, a MacBook Pro, although it gives me 4 good hours of battery life, was not enough for a transatlantic trip. Enters the iPad, with 10+ hours of battery life… in fact, way more, I happen to use it all day for work without recharging, I guess to get to 10 hours only you must run it for viewing movies or gaming - whatever. Anyway, among my needs, a good text processor comes top of the list. Looking over the app store, and googling “iPad Markdown” do not give any tangible result. Jumping to conclusion after a bit more research, there is no Markdown converter/viewer for iPad. Interesting?
Why would that be? There are a string of possibilities, and I went through most of them I caught up 3 that were the most tangible:
1. it’s not easy to implement, so developers who have thought about it have either given up or not succeeded, or considered that the result was not worth the effort
2. you can’t use a script to carry out the conversion, so it’s one of those cases where we need to re-invent the wheel
3. there is no commercial interest for this kind of app…
Let’s start with number 3: I picked up on the web a nice review with stats about the spread between various types of apps in the App Store. Comes out that only 10-12% of app are filed as “Productivity” tools, while more than 80% of the Store is about gaming. Well, given the growth rate of the Store, and the number of devices sold every single month, there is a good potential user base. Even if you’re touching only 1% or 0.1% of the whole park of devices out there, that’s still a huge market.
Number 2, I did not think about straight, it came a bit later, when I was researching about Objective-C and iPhone/iPad programming. Comes out that it is impossible to use a perl script inside an app. On a Mac, not a problem, you define a NSAction and bang you’re done, but not on an iDevice. That’s a bad blow, because it means that you need to program an in-app converter, in C or Objective-C ideally. See, if this was not the case, I could have simply used John Gruber’s Perl script and be done in a breeze… but so would have other devs. Number 1, that’s a worry only when you get into the programming phase, and sweat over yet another freaking parser. Did I mention I hate parsers? Every developer of the planet must do too… Anyway, let’s get to the who-whyme and not be too afraid from all of that.
My main problem here is not to develop or not to develop, it’s to evaluate if this is possible, and whether I’ll have time to go through it. I’m not a developer per se, by which I mean tha tI’m not a computer engineer. I’ve been trained as a Materials Scientist, and took a PhD in the same branch, in polymer physics. I’ve honestly loved it, and still do. Yet, I’ve got my own Company to run, and we’re involved in such a small niche that our distribution is not easy, although developing for these Nanotech processes is really interesting.
Anyway, I’m well trained in a dozen of languages, and since I’m away from the labs now I spend most of my time developing. My two languages of choice are Fortran (I think in Fortran) and Java (I speak in Java), and I mainly use a simple text editor for Fortran programming and Eclipse for Java. All the NIL Simulation Suite is programmed in Java by the way. Like any good scientist, I’ve also had to deal with C, Python and C++, although C++ has always confused me a bit.
Being a scientist can take various meanings, but for me it covers a few essential qualities: you must be patient, and very resilient. As I told my future boss when interviewing for my PhD position: when I bite a bone I don’t let it go. And most of all, you must not be discouraged easily, otherwise Research is going to be very hard for you, as you quickly learn that things are never over.
Armed with this, I can probably learn a new language, Objective-C, in two weeks to a month, and try to develop a parser for Markdown to HTML conversion. Cranky, but no big deal. And it comes at a good time in my personal/professional life: wife and kid on vacation, and I’m definitely getting tired of programming polymer physics, been on that nano-imprint problem for two years day and night, I need a break. It’s either that or programming a 3D Tetris or Rubik’s cube. Got to be honest though, I also researched the possibility to get a 3D visualization engine for Google Sketchup and other CAD formats up and running for this platform, but I’ve been having so many problems with converters/parsers of 3D geometries lately for the NSS that I decided to pause this. Really needed something else to distract me.
For me it’s a go. Buy a few books and read the intro chapters to see where we’re going in terms of technology and implementation. Get real, do a first plan of development and jolt down a few sketches to serve as a guide during the reading/assimilation and maybe development phase. Point is, I’ll be reading and learning selectively the advanced features of the language and the interface once I’m done with the basics of Objective-C. I’ll complete a few strategic points before going further, I’m still way way far from writing my first line of code!
As I said, I had a look for 1-2 hours at the business case, and it looked OK, although I could not pin some precise figures on how much the app would make. But it seems that selling such an app between 5 and $9 has potential. I did a quick SWOT analysis as well, which in general helps me to focus on the essential afterwards. I left all the fuzzy questions such as the definition of a mission statement (I don’t give a shit, that’s Corporate bull) and the implications of not achieving this strategy: I’m not trying to convince anyone but myself, and keep a few notes while I’m going through the creative process. So, simple SWOT with 2 bullets under each category:
1. Strengths: Markdown is really easy to use, and would make a perfect replacement for a rich-text word processor on the iPad. Dealing with text only files (ASCII) makes sense because of the very low memory and storage footprint envisaged;
2. Weaknesses: typing on the iPad, come on!? That’s a reduced keyboard, I can’t find the characters I need for Markdown in less than 3 strokes. Remember how Apple revolutionized the Music industry with the iPod? every single function was available in less than 2 strikes, which I still admire: the device was cool, but the software was a masterpiece in terms of usability. There are a few other weaknesses that could be mentioned, but I went for the fact that when I’m typing text in Markdown, I’m constantly switching between the app and the web-browser to gather links and pictures… which is not possible on the iPad. Well, it’s possible, but it means going out of the app and coming back after having copied a link, it’s not handy at all. And although the app’s got nothing to do with it, it’s still going to piss me off.
3. Opportunities: there’s no other app that is doing Markdown to HTML conversion out there! Tumblr is, in my mind, the best and easiest blog engine out there, and it takes Markdown text as an input…
4. Threats: any guys out there developing the same? Note to myself: keep Googling this entry along the process, check blog entries and discussions to identify key players. There are already a good number of text processors, they may come up with an innovative idea on how to do RTF in a way that complies with the device. Check what the (much admired) Omnigroup is doing, what Marco.org is up to, MarsEdit, the guys who do Coda and CSSEdit… basically keep up-to-date with all the big Guns: they’ve got the experience, the name and respect in the industry and among users - and they well deserve it. With 15 years of expertise in another field, you’re a newborn to this industry, don’t know the codes and rules, better be humble - another fresh start.
That’s all folks
Good way to start, we’re going to maximize our strengths, which are really those of the Markdown language, try minimize our weaknesses, achieve the opportunities and monitor the threats. Worse come to worse, someone else may come up with the same idea and do it better and faster than I, but I will at least have learnt a new language and system, and will be able to use it for something else (got other ideas relating to Engineering - exciting!).
That’s all for now, in the next post I’ll be scribbling on the blackboard my features requirement, which will lead to the development plan. What’s happening in between? Read everything you can, get to know what’s out there, read products reviews, look at interfaces, and get to understand the terms you’re not familiar with in the first chapters of those iPhone dev books.