Our Android app and Samsung’s repeatedly regressing bugs in Accessibility Services

Hello folks,

A number of you, especially international users, are affected by a very annoying bug in Samsung’s build of the Android OS. The unfortunate situation is that this a Samsung bug, and not something we have the ability to do much about. This Samsung bug variously causes these behaviors:

  • Installing RescueTime and enabling “website details” causes Text To Speech to be active. This one seems to be mostly solvable through ridiculously complicated systems settings changes.
  • Installing RescueTime (and enabling website details?) causes misbehavior of certain alternate keyboards, especially Swype. Doesn’t appear to be a solution to this yet.

 

Samsung has at certain times claimed to fix this bug, but it is as if they are using some stub code that contains the bug, and keep re-introducing it in different ways. The bug has to do (it seems) with Samsung incorrectly responding to other apps Accessibility settings, when they should not.

They seem to have introduced the bug in some revision 4.1, then sort-of fixed it in some iterations 4.2, then re-introduced it in other ways in 4.2.1, at this point it is hard to know which Samsung devices have the issue. Galaxy 3 seems to be the biggest offender.

Here is a comprehensive discussion of other app developers hoping to get Samsung to do something about it:

http://developer.samsung.com/forum/thread/samsung-devices-and-accessibility-services/77/204387

and another thread: https://code.google.com/p/android/issues/detail?id=23105

and another about keyboards: http://forum.xda-developers.com/showthread.php?t=1924208

For users with the TTS and Talkback problem: from what we hear from users if you go to your phone’s system Settings -> App -> All and disable BOTH Google TTS Engine AND Samsung TTS Engine, the spoken text problem should go away.

For users with the Swype and other keyboard problems, we are still looking at recommendations, and will update here. Some users may have success by simply switching the RescueTime Accessibility Service to OFF under system Settings -> Accessibility -> Services -> RescueTime (switch to OFF).

Our current plan of action is to add a feature that detects if you are on Samsung devices, and if you select web site details, give you a warning and a link to this post.

Thanks!

 

 


RescueTime handling of Heartbleed SSL bug

heartbleedThis week, a security vulnerability known as the Heartbleed bug was discovered to be affecting major websites across the internet. RescueTime’s servers have been updated to address this issue.

All requests to RescueTime use SSL (HTTPS). All requests are terminated by Amazon using their Elastic Load Balancing Service. This service was patched to eliminate the Heartbleed bug on April 8th. This means users are currently protect against leakage resulting from this bug.

Additionally, as of April 9 all RescueTime server systems have been patched for the bug, or have been identified as not vulnerable. This is more a precaution than requirement since users do not directly connect to any RescueTime servers.

RescueTime is in the process of updating all passwords used in the administration of the service as the dependent services themselves are updated to protect against the bug, e.g. when the site service we use announces they are patched, we then update the password.

However, for further guarantee of security RescueTime will also update its server SSL certificates used in HTTPS and other privileged resources over the next week. We will make a second update when that is complete.

What should you do at this point?

It is now safe to change your password on www.rescuetime.com. You may also want to read our list of general steps you can take to browse the web safely while other websites are responding to the Heartbleed vulnerability.


RescueTime for Android adds Website Reporting

(Firefox users: up vote this if you want support on Firefox for Android: https://bugzilla.mozilla.org/show_bug.cgi?id=908224)

We just pushed an update for our Android app that adds the ability to report on time you spend browsing on your phone or tablet. Get it here:
https://play.google.com/store/apps/details?id=com.rescuetime.android

To do this, we needed use Android’s Accessibility services, and this requires an elevated privilege you will need to manually enable. Our app will walk you through this when you click the “Enable website logging” option. However, here is a brief explanation of the process:

1) Open up the RescueTime app and click the settings button (the gear icon). Click the “Enable website logging” option. This will automatically take you to the system Accessibility Settings screen, if it needs to be enabled.

RescueTime Settings

2) Find RescueTime in the Services list on the Accessibility settings screen and select it. On older devices you may already see an “on/off” switch for RescueTime here, just select On and you are done.

Screenshot_2013-08-16-10-38-53

3) After tapping it, on newer devices it opens the screen for enabling the service for RescueTime that has a description of the service. Click “On” to enable it. This automatically signals RescueTime to begin looking for site info in browsers.

Screenshot_2013-08-16-10-39-01

4) Achieve success! Supported browsers are: the stock Android browser, just called “Browser”, the Nexus series stock browser (a version of Chrome), Chrome (the version in the app store), Chrome Beta, and Dolphin. Not supported: Firefox and DolphinMini.


Public Service Announcement: Old “Lite” browser plugin has been sunsetted

A really quick sunset, the kind you see in the tropics. REALLY quick. I’m thinking: tomorrow. This being the kind of sunset where no new data is accepted from these old client apps.

We have new plugins for both Firefox and Chrome that replace the old. They have been out for quite a while now, and the old one has been de-listed for a long time. Here’s where the new one is (links to extension galleries):

https://www.rescuetime.com/browser-plugin

I imagine this affects no actual person, only zombie systems that are enjoying harassing our site, but if you are a person or sensitive “good” zombie currently using the old plugin, please switch to the new one.

IF you are an old plugin user, you can follow these steps and keep your old data:

1) Open the full dashboard on our site from the plugin: https://www.rescuetime.com/dashboard

2) Click “settings” top right and set an email address for yourself, and add the password

3) Delete the old plugin from your add ons/extensions list

4) Add the new one https://www.rescuetime.com/browser-plugin and register using that email address


Kindle users: RescueTime is now available on the Amazon Appstore

kindle-fire

To all Kindle Fire users: just a quick post to let you all know you can now get RescueTime for Android directly from the Amazon Appstore without having to go through alternative marketplace hoops.

RescueTime for Android works by noting how long you spend in your mobile apps and phone calls, reporting back to you your efficiency score, top distractions and categories right on your mobile device. There is a handy stopwatch tool for manually tracking things like meetings and exercise, and you can set a productivity score for each activity as you log it.

If you also have the RescueTime desktop application installed, you’ll be able to see your mobile time right alongside your other logged time:

mobile-rescuetime

Here’s the listing: RescueTime for Kindle Fire

Of course, users of non-Kindle Android devices can still get RescueTime for Android from the Google Play store.

Enjoy!


RescueTime adds PayPal subscription support

After many requests on behalf of customers– especially users outside the US, we are happy to announce we support subscription to premium service using PayPal.

Screen Shot 2012-12-10 at 10.32.32 AM

PayPal payment choice is available at signup time or from the billing page for existing accounts. You can upgrade from free plans, or convert from credit card payment– it should support all account transitions.

Teams can pay using PayPal as well. Because the subscription is managed by PayPal and requires your approval for each change, to add or remove seats you need to go through a few extra steps than when using a regular credit card, but any plan is supported.

New users: Get RescueTime using PayPal

Existing users: Upgrade RescueTime using PayPal


Build it and they will come? Performant Search Part 2: The Technology Sauce For Better Spaghetti

Our job was to find a long term scalable solution to the problem of finding your activities that match your key word search. This post pertains to the technology involved. Read about product features and new capabilities here.

Turns out, search in RescueTime is a surprisingly complicated problem, for the simple fact that your typical search prioritizes ranked relevance results– it’s ok for the engine to stream results to you, it’s ok for the search to be incomplete, as long as the ranking is bubbling up the best match. It’s ok for it to be probabilistic or best-guess in nature, sometimes. Generally speaking, you are looking for something small (some words) in something large (a web page).

Our challenge, is that while the user experience semantically matches “search”, what you really need is a complete result set, combining all relevant reporting filters, of activities that match your requested search expression. It should produce repeatable results assuming new data hasn’t arrived. It should be real-time updatable as your new data comes in (~ every 3 minutes). It should be ok if every record or zero records match, there can be no cap on number of matches. All this, for about 100-400 logs of time data per user per day for many tens of thousands of users. The longer a user is with us, the huger the list of activities to match against, just for that user. The list of unique activities of our users is well over 1 billion. We should be able to completely rebuild indexes of activities at will, in near real time, to provide application improvements. Yet, cost needs to scale at worst linearly as a fraction of the rest of our products’ demands, and speed needs to remain constant or better.

All of these characteristics eventually killed our previous attempts to use industry-standard search models based first on Lucene/solr, and secondly on Sphinx. Both were able to hold up for a period of time, but were fundamentally flawed solutions in the assumptions they make expecting a relatively static, ranked-match document-based search model. To shoehorn them into our needs required operational spaghetti and some pretty ugly reporting query structures.

Our old search platform may have looked like this and required similar engineer attention, but it didn’t sound as good.

Enter MySQL fulltext Boolean search. First there is the critical advantage of being *inside* and native to our database platform in a way that even Sphinx with it’s plugin can’t be. This allows for more integrated and simpler reporting queries– no longer is the step of matching your search expression required to be isolated from the reporting query that depends on it (Sphinx could have done this, sort of, with the plugin, but not really the same). Second, in Boolean search mode, MySQL provides an unlimited result set (no cap on results). Additionally, there is less duplication of supporting data, since it can operate entirely in the same instance as the source data– this value is not to be underestimated, for all the inherent caching this leverages. Operationally, it is far easier to dynamically and programmatically add, destroy, and rebuild indexes– since they behave like tables with normal indexes to the operator.

But for performance, the  most critical options it offered was a way to fluidly and transparently provide per-user-account search indexes, which lets our performance remain consistent despite constant multi-dimensional growth (new users + accruing existing users’ time data). This isolation-of-index solution would have been theoretically possible but horribly unwieldy and in need of huge operational supporting code in the other options. Secondly, it provides a clear way to constrain the size of keyword indexes, in other words, we know from your requested report you could only possibly care about activities that were in the particular time range you requested, and this can be of value both in index partitioning options and the submitted report query itself, especially in the amount of memory that must be held to build final results. A huge benefit of this known-maximum-scope for the searchable data means that at any time we can intelligently but programmatically throw away or update whatever dynamic table plus index we had that intersects the requested scope rather than the entire source tables, and rebuild it in real time, for a minor speed penalty (< 0.01 sec vs .1 to 3 sec for your search). Any subsequent search request that remains a subset of that most recently persisted scope can just reuse the current index, with the < 0.01 sec speed. We can play with permitted scope expansion to tune for speed. Furthermore, any sharding of accounts across new instances allows the search data to naturally follow or be rebuilt inline with the same scaling decision that drove the sharding to begin with– no separate stack to worry about following the shard logic.

Check out some example code for sneaking a search result into a series of joins rather than hanging out in the WHERE clause. Here it can be treated just like any other join, and like any other filter on your pivot reporting.

[sourcecode]
– check dynamically_maintained_text_and_things_with_controllable_scope
– if exists and is up to date continue,
– else, create if needed and
– push missing scope by intersecting request scope with existing
– index is maintained by table definition

SELECT * from things
INNER JOIN other_things on things.id = other_things.thing_id
– begin search acting like join
INNER JOIN (
SELECT * FROM ( SELECT things_id
FROM dynamically_maintained_text_and_things_with_controllable_scope `things_search_data`
WHERE MATCH (things_search_data.a_text_field, things_search_data.another_text_field)
AGAINST (‘search words here -there’ IN BOOLEAN MODE)
) d_things_result_table
) things_result_table
ON things_result_table.thing_id = other_things.thing_id
– end search acting like join
WHERE
other_things.limiting_characteristic = a_factor
[/sourcecode]

We’re using the table alias for the search data there because that would allow you to chain multiple searches, one after the other (search inside a search), against the some source data by adjusting its table alias.

Engineers looking to provide high performance, easily maintainable search capabilities should carefully consider MySQL fulltext search options if they are on a late version of the product. This is especially true if you are trying to manage a search index of billions of tiny items based on database text columns that mutate very quickly, and benefit from programmatically created and maintained indexes. For example, it occurs to me that Twitter could use this model in very much the same way we do to provide some powerful realtime search indexes for targeted subsets of accounts or hashtags.

In our case, we were able to reduce our search related platform costs by about 60-70%, significantly reduce our operational pain, even while delivering a solution that provided vastly improved performance and eliminating that “well, it works now but we’ll just re-architect next year” problem.

Our spaghetti search code and virtual spaghetti cabling has now been re-factored into something tastier, ready for your consumption.

creative commons credit to Eun Byeol on flickr