
Remote code execution vulnerability in SQLite - LinuxBender
https://blade.tencent.com/magellan/index_en.html
======
nneonneo
It is _very_ likely that this bug only affects systems which accept and run
arbitrary SQLite3 queries. This includes Chromium, because Chromium ships with
WebSQL. The Google Home is probably vulnerable because it can be coerced to
load a webpage. I doubt that this bug affects systems that merely use SQLite
as a database without providing external query access.

My best guess for the bug is that arbitrary SQLite queries, prior to 3.26.0,
were permitted to write to the shadow tables used by various plugins to
implement features. fts3/4, prior to 3.25.3, appear to contain an integer
overflow bug which can be triggered by manually modifying the fts index data.
A careful application of this integer overflow appears to make it possible to
truncate a writable buffer, leading to a nice heap overflow condition that can
be exploited by further crafted SQL queries.

The primary integer overflow bug was fixed in
[https://sqlite.org/src/info/940f2adc8541a838](https://sqlite.org/src/info/940f2adc8541a838)
"Add extra defenses against strategically corrupt databases to fts3/4.",
committed as part of the 3.25.3 update (which is what Chromium updated to).
Later, in 3.26.0, they further secure it by making shadow tables _optionally_
read-only.

The worrying thing here is that SQLite3, in its default configuration, is
still not convincingly secure. Being able to write arbitrary data to the
shadow tables has the potential to break all sorts of assumed invariants, and
it's pretty clear that the SQLite3 developers did not necessarily anticipate
all the ways in which this could break. The "SQLITE_DBCONFIG_DEFENSIVE" option
which was added does not appear to be on by default, and it breaks backwards
compatibility (setting it causes SQL imports from .dump to fail because .dump
assumes shadow tables are writable during import).

There may be more bugs lurking in this area - this would be an _excellent_
opportunity to fuzz all the plugins in SQLite to see if any of them barf when
their shadow tables are corrupted.

~~~
SQLite
Excellent summary, nneonneo. I think everything you said here is correct.

The vulnerability only exists in applications that allow a potential attacker
to run arbitrary SQL. If an application allows that, it is usually called an
"SQL Injection" vulnerability and is the fault of the application, not the
database engine. The one notable exception to this rule is WebSQL in Chrome.

I put up
[https://www.sqlite.org/security.html](https://www.sqlite.org/security.html)
recently to serve as guidance for people who want to live on the edge and give
unrestricted SQL access (or unrestricted database file access) to potentially
hostile attackers. That page is a work in progress. More could be said. For
example, it is probably also a good idea to use various obscure APIs to limit
the length of SQL statements or the amount of memory that can be used, to
avoid DOS attacks. I'll keep improving the document as I have time.

Our intent is that SQLite _should_ be secure against these kinds of attacks.
We have spent years fuzzing it to try to find these problems. But the thing
is, we never configured a fuzzer in such a way that it might start modifying
the shadow tables of FTS3, and so we missed this one. Moral: never
underestimate the ingenuity of a motivated gray-hat.

The Chrome people have recently starting fuzzing SQLite database files on
Google's infrastructure. We had previously only fuzzed database files on our
own workstations. It's amazing the number of new problems you can find when
you run a fuzzer at scale. :-) A few more problems have been fixed. We are not
aware of any exploits. And in particular, if you follow the advice of the
article above and "PRAGMA quick_check" untrusted database files or set "PRAGMA
cell_size_check=ON" then none of the recently found and fixed issues are
reachable.

~~~
kd5bjo
If giving unrestricted database file access is “living on the edge,” maybe
[https://www.sqlite.org/appfileformat.html](https://www.sqlite.org/appfileformat.html)
should be updated to reflect that?

~~~
zzo38computer
I agree. If the file is one you might get elsewhere rather than only being
your own files, then it is untrusted.

To me, "trusted" is: queries entered by the local user (or, for setuid
programs, by the local system administrator instead of the local user), or
that are built in to the program. Others are untrusted.

And yet, I have already considered these kind of vulnerability before even
knowing about it.

------
modeless
SQLite is the most thoroughly tested codebase I'm aware of [1]. It has seven
times more test code than non-test code. 100% branch coverage. If even SQLite
can have a RCE vulnerability, I'm convinced that it is not feasible for
_anybody_ to write safe C code.

[1] [https://www.sqlite.org/testing.html](https://www.sqlite.org/testing.html)

~~~
segmondy
100% branch, line coverage means nothing. It's about logical coverage. What
are you testing for? You are not testing lines of code, but logic.

~~~
yladiz
Isn't that the definition of branch testing, to test all possible branches
within code and also testing the logic in all of those branches?

~~~
gregmac
Consider a line like:

    
    
        value = 1 / input;
    

You can get "100% coverage" if you test that with `input = 1`, but unless you
check with `input = 0` you're missing a quite important logical check.

~~~
breckuh
Can't you just have a NonZeroInt type?

~~~
whyonearth
Only if your language supports dependent types.

~~~
tonyg
It's perfectly possible in languages with ordinary ADTs.

    
    
      data Nat = Z | S Nat
      data NonZeroNat = OnePlus Nat
      data NonZeroInt = Negative NonZeroNat | Positive NonZeroNat

~~~
jwatte
C isn't one of those languages, though.

~~~
tonyg
The analogous would be the Go-style represent-a-sum-badly-as-a-product,

    
    
      struct nonzero_t {
        int is_negative;
        unsigned int one_less_than_the_absolute_value;
      };
    

which, under interpretation, ranges from -(2^32) to -1 and +1 to +(2^32).

------
coleifer
My guess, for what it's worth, is that chromium is vulnerable because it
exposes sqlite to web applications, which can then execute queries in such a
way as to achieve code execution.

I highly doubt this would affect, say, a blog running with a sqlite database.
From the alarmist nature of this post, though, it's unclear.

I see chromium, in their patch, switched to using new flags when opening the
DB. There are also some sqlite changes that seem to prevent meddling with
virtual table shadow tables (eg inverted index for the fts3 extension).

The question I think everyone is asking is how much sqlite needs to be exposed
by an application in order to be vulnerable?

Just my thoughts. Eager to learn more.

~~~
tptacek
Not so much as a guess as a documented fact:

[https://chromereleases.googleblog.com/2018/12/stable-
channel...](https://chromereleases.googleblog.com/2018/12/stable-channel-
update-for-desktop.html)

The Chromium exposure is through Web SQL.

~~~
tedunangst
That doesn't really answer the question about other applications. What SQLite
function do I need to not call with network input?

~~~
wglb
See
[https://news.ycombinator.com/item?id=18687184](https://news.ycombinator.com/item?id=18687184)

------
blattimwind
An RCE in a non-networked component is interesting (in other words, obvious
hyperbole). Either this is your usual corruption bug/vuln triggerable in
some/many programs using SQLite, or an actual bug in SQLite itself, e.g. query
preparation (a fix/workaround being committed to SQLite doesn't necessarily
imply one or the other). Whether the RCE hyperbole is justified remains to be
seen.

Edit: Apparently the exploit vector is due to WebSQL.

And my guess as for the vulnerability area are "strategically corrupt
databases", because there have been numerous commits related to this in the
relevant SQLite releases and some seem like they were added relatively late in
the process (e.g. after changing the VERSION file but before releasing).

~~~
JaRail
They mention the example of browsing to a web page. Not quite a fully fledged
remote execution bug but close enough. I think what they're really saying is
they're aware of remote execution cases. An example of that might be a web
services that's backed by a SQLite query.

~~~
tptacek
Drive-by code execution from a page in a web browser is almost the textbook
example of "RCE" in its modern sense.

~~~
GalacticDomin8r
Yeah, of the browser or plugin. SQLite is neither although does have a local
exploit.

------
rasz
Ancient and long dead Opera 12.xx let you set all HTML5 Offline Storage
quotas/features globally and per domain.

Afaik Chrome and its derivatives lack any form of user control over local
storage. No Quota mechanisms, no domain black/white listing, no feature
toggle. localStorage, webSql, IndexedDB, Filesystem API, all forced on with no
limits under user control.

In a better world this would be an easy fix for Chrome users unable to upgrade
their browser, flip one config setting to disable webSql and you are done,
alas Google wont let you do that. Cant wait for first worm using this vuln.

------
jared0x90
I casually thumbed through a few of the commits they posted and came across
this

[https://chromium.googlesource.com/chromium/src/+/c368e30ae55...](https://chromium.googlesource.com/chromium/src/+/c368e30ae55600a1c3c9cb1710a54f9c55de786e%5E%21/#F48)

    
    
       for(i=0; i<nChar; i++){
         if( n>=nByte ) return 0;      /* Input contains fewer than nChar chars */
         if( (unsigned char)p[n++]>=0xc0 ){
       -      while( (p[n] & 0xc0)==0x80 ) n++;
       +      while( (p[n] & 0xc0)==0x80 ){
       +        n++;
       +        if( n>=nByte ) break;
       +      }
         }
       }
       return n;
    

Looks like there may have been an issue in parsing malformed multibyte unicode
characters properly.

~~~
jared0x90

       const secondStatements = [
       "SELECT quote(root) from ft_segdir;",
       "UPDATE ft_segdir SET root = X'0005616261636B03010200FFFFFFFF070266740302020003046E646F6E03030200';",
       "SELECT * FROM ft WHERE ft MATCH 'abandon';"
       ];
    

Just saw the proof of concept page. Looks like they are building quite the
usual string in hex... Starting with a null terminator? Mmmhmmm

------
saagarjha
Unfortunately this announcement is light on details; does anybody know what
the actual vulnerability was?

~~~
testplzignore
Maybe related to
[https://www.sqlite.org/src/info/8576ccb479fc4b76](https://www.sqlite.org/src/info/8576ccb479fc4b76)
?

Edit: Yes, this is probably it.

~~~
tedivm
That commit seems to imply that it's a bug when running on Windows, but the
Tencent Blade folks have said they've exploited this on Google Home devices.
My guess is that this commit is one of many that helped resolve the
vulnerability.

~~~
robocat
"Google Home" is either:

the app installed on your phone to control say a Chromecast,

or a smart speaker.

------
libdjml
Remote implies it can be accessed remotely, is that true, or did they mean
“remote if an attacker can remotely send data to SQLite”?

~~~
haney
Sounds like they mean “remote” because chromium uses SQLite and JavaScript
loaded into your machine comes from a remote source. So because a website can
run JS that can exploit chromium they’re calling it an RCE.

~~~
libdjml
Yeah it seems like RCE in the context of Chromium, but not SQLite? I know it’s
pedantic but if this is RCE in SQLite because it’s exposed to the network via
other software, every vulnerability is “remote” because you may expose it via
other software.

------
jondubois
This is surprising considering that SQLite is very heavily tested. It shows
that ridiculous amounts of testing with 100% coverage of every code path and
"millions and millions" of test cases still doesn't guarantee that the program
always works as intended.

I think that this is an important lesson about testing. We should have fewer
tests but we should try to get the most value possible out of each one and for
developers that means actively seeking out unusual edge cases that are likely
to break things.

Source:
[https://www.sqlite.org/testing.html](https://www.sqlite.org/testing.html)

~~~
SQLite
Two points:

(1) The coverage testing used by SQLite is very good at finding problems that
occur when the system is used as it was intended. Fuzz testing is better for
finding vulnerabilities that can be exploited by a hacker. The 100% MC/DC
testing in SQLite is very useful in ensuring that the code does what is
intended for sane inputs. And 100% MC/DC helps prevent us from breaking things
as we evolve and enhance the code. But the MC/DC testing is less useful at
fending off attackers.

(2) The magellan vulnerability exploits a bug in an SQLite extension, FTS3,
which while very well tested, is not testing to 100% MC/DC. (See the second
sentence at
[https://www.sqlite.org/testing.html#test_coverage](https://www.sqlite.org/testing.html#test_coverage))

Hence my takeaways from this episode include that I need to extend 100% MC/DC
testing to all commonly used extensions in SQLite, including FTS3, FTS5, and
RTREE, and I need to improve fuzz testing throughout SQLite but especially in
extensions.

Advocates of "safe" language correctly observe that this particular problem
would not have happened if SQLite were written in (say) Rust. Rewriting SQLite
in Rust in not (yet) a viable solution. (See
[https://www.sqlite.org/whyc.html](https://www.sqlite.org/whyc.html)) But I
can start moving SQLite in that direction, and perhaps make use of techniques
taken from safe languages to improve its resistance to attack.

~~~
mwhicks1
Hopefully soon, “moving in that direction” can be done by slowly porting to
Checked C, while always retaining an executable artifact.
[https://github.com/Microsoft/checkedc](https://github.com/Microsoft/checkedc)

------
sat_nam
Zhuowei Zhang (@zhuowei) published a proof of concept that crashes Chrome 70:
[https://worthdoingbadly.com/sqlitebug/](https://worthdoingbadly.com/sqlitebug/)

------
simonw
Python ships with a sqlite3 module in the standard library.

Does this mean Python needs to ship a security path? What should Python users
be doing about this?

~~~
sdinsn
But does the sqlite3 module actually contain SQLite with it, or just a library
to interface with it? The fix does not change any interface library code.

~~~
armitron
On Windows, a complete sqlite3 DLL (~1MB) is included with the Python
distribution.

On Linux/macos, the Python extension (usually) links dynamically to a shared
sqlite3 system library.

------
rhexs
"If you use a device or software that uses SQLite or Chromium, it will be
affected."

If I write a hello world C program that does some sort of IO with SQLite, it
will be vulnerable to remote code execution? (if this turns out to be true,
that will be quite impressive!)

Guessing something was lost in translation there. Sounds more like someone
found a way to get code execution if you can inject certain data into SQLite,
then found various applications that expose this functionality remotely?

~~~
sebazzz
Probably they found the vulnerability through Chromium, then extended that to
"everything that uses SQLite". Hard to tell anything without more details
though.

But if that is the case this is _huge_. SQLite is used in many places
nowadays: Websites, browsers (Chromium and Firefox, I know of), various
software including some Android apps. That also probably means the attack
vector is some procedure where input is sanitized (assuming SQLite provides
that, I never programmed against the C API).

~~~
ronsor
WebKit; worse off, WebKit and Chromium expose SQLite almost directly through
WebSQL! Drive-by malware!

~~~
jakeogh
If I understand correctly, it requires JS to be enabled (which it is usually).

(Edit: wrong term, it's not "HTML5 Local Storage", it's "HTML5 Database"
thanks):

Chromium (EDIT: idk yet)

Webkit2:
[https://webkitgtk.org/reference/webkit2gtk/stable/WebKitSett...](https://webkitgtk.org/reference/webkit2gtk/stable/WebKitSettings.html#WebKitSettings
--enable-html5-database)

~~~
ronsor
Drive-by malware used to require Flash or Java... which used to always be
enabled.

Edit: don't disable local storage! you'll break lots of things that way, and I
don't think that includes WebSQL.

------
zzo38computer
The question are: Where is the vulnerability? By executing user-specified SQL
statements (with or without setting an authorizer callback; I have once
reported a bug causing SQLite to segfault in some cases when the authorizer
callback denies something)? By downloading a corrupt database? In some
extension (if so, in what extension)? In the VFS? What circumstances are
needed to exploit this?

------
Buge
Huh, so Tencent Blade is like Google Project Zero? Ideally all companies would
start attacking each other, and improve everyone's security.

------
miketery
Pretty sparse on details. I presume local SQLite files for backend systems are
unaffected, can anyone confirm?

------
odc
Any news about Firefox? Are they also impacted?

~~~
fbender
Apparently not since they resisted implementing WebSQL.

------
klarrimore
Pretty light on the details here.

------
craftoman
When people will start realizing how fragile is actually every project from
Google. In the end it's built by humans, remember past week about kubernetes
and consider the fact that Google pays thousands of dollars to white hackers
who helped them in the past.

