Hacker News new | past | comments | ask | show | jobs | submit login
Psd.rb (layervault.tumblr.com)
629 points by chamza on July 30, 2013 | hide | past | favorite | 92 comments



This programmer has written about the PSD format in colorful detail: "Trying to get data out of a PSD file is like trying to find something in the attic of your eccentric old uncle who died in a freak freshwater shark attack on his 58th birthday."

https://code.google.com/p/xee/source/browse/XeePhotoshopLoad...

(ref: first link in the article)


Several years ago while writing a PSD parser in javascript, that comment block saved my sanity.


I think that comment itself was posted on HN or somewhere -- I know I've seen it before.


It was a front-page story on HN, but four years ago: https://news.ycombinator.com/item?id=575122


Thanks for the link. I still remember that thread and the original post. It was indeed awesome.


>Adobe has never produced an easy way for developers to work with the format.

That's not entirely fair. Adobe has openly released a comprehensive description of the format which is, as far as I know, accurate. The problem is that the format itself is a heap of features piled on year after year with apparently no regard for doing things consistently.


The file spec released by Adobe (http://www.adobe.com/devnet-apps/photoshop/fileformatashtml/) is actually outdated, wrong in some places, and can be incredibly vague at times.


I can't speak to its accuracy, but, June 2013 wasn't yesterday. It isn't that old -- and it isn't like there's been a release of Photoshop in the past month and a half, so, it seems pretty recently updated.


recent publish date != up-to-date content


A bit off-topic, but is there a spec for Illustrator files? It seems the only one I see on Google is from an old version of AI...


As noted, newer illustrator files are just PDF's (perhaps with extra data for editing).

Older illustrator files are in fact, EPS files. This is somewhat tricky since, an EPS is not actually so much a data format as it is a turing complete programming language. Sooo... yeah, who knows what black magic they did to pull off reliably reading and writing it.


You can read an EPS file by interpreting it and remembering the shapes it renders on the page.

You can write an EPS file by emitting your list of shapes without using the turing-complete features of the language.

Not that hard, at least conceptually.


That's all fine and good at least up until you have to kern a line of text. Then what?


Not yet, but there's always next week...


Some AI files can read as PDF (try to rename them and see if your reader can open it).


joke: Legal wanted to open-source it, but was vetoed by Engineering.

But seriously, this kind of back-compatibility horror is the hallmark of success, where success means enduring.


Similar to a Word .doc? Who's worked with both, is it Better, or worse?


Wow, fantastic work and a lot of respect for open sourcing this lib when it's clearly an important part of LayerVault.


Does anyone else think it's weird that they decided to make this library in Ruby? It drastically cuts down on the audience. Why not C/C++ with wrappers for all the dynamic languages?

EDIT: nevermind, it makes sense now that I see their main product is a version control system for designers. Still, it would be nice to see this ported to native code some day.


Most of the work is done, get off your ass and start porting.


I might just accept that challenge!


as a programming language fan, i've never found anything as productive as ruby. it may not be the fastest language out there, and it may not have a bunch of static safety guarantees, but for exploratory programming it's pretty much unparalleled. if i were tasked with making a psd reader, i'd definitely do it in ruby first and then port it over to whichever language the client called for.


I just wrote a script with this that takes a directory of PSDs and outputs PNGs for each one. It took about 2 minutes. This is great.


Great, however, that was possible with ImageMagick before. Even more: ImageMagick can extract PSD layers as separate PNGs.


Sure. The difference is that with ImageMagick is getting all the flags right. Not to put down a powerful tool, but it isn't exactly memorable. In a script you can describe what's happening and customise it at any point without in-depth knowledge of the tool, and with a friendly CLI environment. I'd rather write 5 lines of Ruby than read up on the ImageMagick flags.


I'd rather look up the ImageMagick flags and write a simple 5 line shell wrapper for my specific use case than learn an API and write Ruby.


Could you provide your ImageMagick command, so we don't have to navigate the flag-infested waters?


How long did it take to output each PSD (if you have a rough file size for each that would be great to know)


Wow, everything old, is new again ;)

I think with an actual copy of Photoshop, and a little Applescript, this is something you could have done > 15 years ago.


But this way I don't have to keep paying $20/mo to Adobe CC so that I can run the script again next month.


I suspect that you could probably also manage it with GIMP/guile, but I don't suspect it would be particularly pleasant.


If you want to script it, ImageMagick or GraphicsMagick are probably a better bet than trying to hack up something with GIMP batch processing: http://www.imagemagick.com/www/formats.html

If you just want to do basic conversion ignoring layers, it's quite easy:

   for f in *.psd; do
     convert "$f" "${f%%.psd}.png"
   done


You can use find/xargs for this. It'll be faster because you can parallelize it with the -P option, for example, for a 4-core machine:

    find . -iname "*.psd" -print0 | xargs -0 -P 4 -n 1 -I I convert I I.png
For best results, change the number "4" in the above command to the number of cores you have.


The same with GNU Parallel, only it picks the right number of cores automatically and removes .psd from the file name:

  find . -iname "*.psd" -print0 | parallel -0 convert '{}' '{.}.png'


Oh excellent, I didn't know ImageMagick did PSD.


Gimp's support for PSD is understandably poor for the more advanced/modern features, so I wouldn't rely on it without being able to compare the output to something else. I use Apple's Preview utility to manually compare results.


So maybe what's great is avoiding the cost of Photoshop, which many have found to be outrageous.


  $ pacman -S applescript
  error: target not found: applescript
  $ pacman -S photoshop
  error: target not found: photoshop
  $ ruby --version
  ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64-linux]
Hm, which of those can I use?


Yes, but now you can do it without having a copy of Photoshop.


Is this more feature complete (esp. regarding to newer PS versions) than e.g. libpsd?

http://sourceforge.net/projects/libpsd/


libpsd and PSD.rb both have features that the other doesn't have. For example, PSD.rb parses text/font data, while libpsd can handle images with zip compression.

libpsd was actually a great reference in building PSD.rb, especially since it was correct during the times that the actual file spec was wrong and more explicit in the type of data being read.


Idea: Use this to dynamically "compile" photoshop files into png, jpgs, etc on the rails assets pipeline.


Thank you very much for doing this! I wish all people hacking the PSD format would join forces and help with one project. There are too many partial implementations which scratch an itch instead of trying to be a full implementation.


I get this feeling with many open source projects, but merges happen much less often than forks.


Does anyone know of a similar tool for Python?

Not trying to start a Ruby/Python fight or anything; Python just happens to be my preferred language.


A shameless plug: you can give https://github.com/kmike/psd-tools a try.

I've read PSD.rb docs and a bit of its code; the implementation is one of the best and complete I've seen (I've checked almost all PSD reader implementations some time ago).

But it seems that psd-tools is mostly on par with PSD.rb. It also have some features that PSD.rb doesn't have, e.g. full support for 'zip-with-prediction' compression, including 32bit layers. Such images are very common in practice, and parsing them is not easy because the compression format is not documented anywhere, and "zip-with-prediction" for 8 and 16bit layers is totally different from "zip-with-prediction" for 32bit layers (for 32bits it is really tricky).

If PSD.rb authors are reading this, I urge them to check the decompression code in psd-tools (https://github.com/kmike/psd-tools/blob/master/src/psd_tools...) or in Paint.NET PSD plugin (http://psdplugin.codeplex.com/) to not waste the time.

psd-tools also knows how to export individual layers, and there is an experimental support for exporting layer groups; it seems that this is not implemented in PSD.rb yet.

PSD.rb has some features that psd-tools doesn't have, e.g. it parses "Font data" which is really cool and hard because the format is not described anywhere.


I checked out psd-tools and it's good. Any chance of you adding their features to yours or vice versa? I know I know, I should do it myself and do a pull request, but just asking if you are planning to?


Unfortunately I'm currently very busy with other projects, so I probably won't implement PSD.rb features myself anytime soon.

I'm trying to provide feedback for psd-tools pull requests, merge them and release new psd-tools versions in timely manner; the testing suite also helps here, so you know, pull requests are welcome :) Most improvements over last 6 months came from pull requests submitted by other great people.

I think that the "reader" part of library is feature-complete. psd-tools reads all the information, but it doesn't decode all Photoshop data structures (some of them are available only as binary blobs). So I think implementing a PSD.rb feature will most likely involve checking PSD.rb code and decoding a binary blob (already loaded to memory) to a Python data structure.


I didn't check your codebase (I just used the library for a few projects a while ago), so you don't have to answer. But if you want to enlighten others as well as me; am I right in thinking that you have a reader while reads the file and then have a 'decoding module' for every blob. So it would be rather straight forward to port from Ruby such a decoding part and plug it into your library?


Yes, that was the idea.

The whole process is divided into 3 stages: reading, decoding and providing "user-facing API":

- on "reading" stage PSD file is read and split into binary blobs (I think this part is done);

- on "decoding" stage "decoding modules" are called for each binary blob; decoding modules should produce Python data structures that closely resembles internal PSD format;

- on "user API" stage decoded data is converted to more convenient format that is easier to work with (e.g. this include building layers hierarchy, and the PSDImage/Layer/etc classes).

I hope that providing new decoders will be rather straightforward, and it seems to work this way so far: contributors haven't touched "reader" part, and I haven't touched it for a while as well. But software development is hard, so we can never be sure :)


Nice! We should work together.


Thanks!

Feel free to steal anything: test PSD files, etc.

If you want to implement layer blending in PSD.rb (required if you want to be able to export layer groups as images) then check libpsd C library - I think it has the most sophisticated open-source PSD blending implementation available.

Your PSD.rb and https://github.com/layervault/psd-enginedata libraries are very nice too; unfortunately I'm busy with other projects now, but maybe somebody will take effort and port PSD.rb features to psd-tools, or I could do this in future - the LayerVault's code will be very helpful.


Just use Ruby for this. I prefer Ruby, but there are times I will pick Python instead because it has better libs for what I need to do, e.g. numerical and scientific libs.


"Just use Ruby" isn't always the best option; if you have a large base of Python code, it's probably preferable to use Python directly rather than using the subprocess module.

Also, a port would be fairly straightforward. Some combo of the struct/ctypes modules (depending on how complicated the data structures are) would make a transliteration pretty simple.


A "large base of Python code" would not be my main reason for eschewing other languages:

- Unfortunately psd.py doesn't seem to exist. Tell me, what is less effort: Implementing that, or building a small app/service that performs the functionality you need using existing tools?

- Also I'd consider the suitability of a given technology before this too, e.g. on my current project there is a lot of Ruby code, but it's a major bottleneck in one of our services performing a particular task. We rewrote that service in Clojure and it performs orders of magnitude faster and uses orders of magnitude less memory.


> Unfortunately psd.py doesn't seem to exist.

This is just plain wrong. There are many Python readers for PSD format:

* https://github.com/kmike/psd-tools

* https://github.com/jerem/psdparse

* https://code.google.com/p/pypsd/

* PIL and Pillow can read PSD files.

I'm biased because I wrote psd-tools, but I think psd-tools API is quite simple, and it has some features that PSD.rb doesn't have, e.g. it supports more PSD compression formats (actually, all of them) and can export individual layers and layer groups as images. It also has more tests than PSD.rb and more PSD files in testing suite.


Nice. Obviously I didn't look too hard hence "doesn't seem to exist". My main point was against reimplementing something already extant just because you have a swathe of code in a given language.


Your point applies more to PSD.rb then - there was a lot of PSD readers in Python, C, C++ and C# for ages :)

But it seems that reimplementing worked very well for them: API, language and a good timing really matters.


> Tell me, what is less effort: Implementing that, or building a small app/service that performs the functionality you need using existing tools?

I think it depends on your use case. In certain cases, you want to do work inline. In other cases, you want to do PS manipulation in the background. I could imagine projects that go in either direction. If I had a large Python desktop application, I'd port the code. If I had a Django application that wanted to do background processing of PS documents, I'd put the work on a queue and have Ruby take care of the messy bits.

> Also I'd consider the suitability of a given technology before this too, e.g. on my current project there is a lot of Ruby code, but it's a major bottleneck in one of our services performing a particular task. We rewrote that service in Clojure and it performs orders of magnitude faster and uses orders of magnitude less memory.

I take a different approach. If I find a bottleneck in Python code, I use ctypes or write a C extension. I'd do the same thing in Ruby. I'm hesitant to deploy multiple runtimes, since I worry about security updates, etc. If I were on JRuby, Clojure would be a natural fit -- I'd only be deploying one JVM.


yes. this https://github.com/jerem/psdparse


Close: https://github.com/migurski/Blit


You can access ruby code from python ( throuh jython and jruby) and the process is relatively seamless.

or check out psdparse on github ( there are others out in the wild )


I would've liked to see it in Go, but alas.


Great work and thanks for building this. There's a lot of room for improvement in automating manipulative photoshop exports, and I look forward to seeing what people do with this, especially in terms of building command line tools.


I have been trying to do this from last 3 months. But I intend to make modifications to layers (turn on or off, change colors) and export to PNG. When I saw Psd.rb I thought it is done. But it just exports the channel data saved by Photoshop. So our modifications wont reflect in it.


Use extendscript or the creative suite extension builder SDK.


Unfortunately I want it to run on a Linux server and I don't want any UI as it needs to be a service.


I reckon a good tool (that this psd.rd wouldn't actually help write, but), would be one that rendered an html/css page - and exported a layered PSD, so you could prototype an app, render it to .psd and send it to your designer friend to make it look rad.


Thank you! There's quite a ton I can do to this. Will definitely be contributing soon.


This is a great example of why open source is brilliant. Everybody wins. We get a PSD parser. LayerVault benefits from the world improving they're core product too.


I feel like this could be used to automatically convert a simple .psd web layout to non-shitty HTML and CSS.


Look up markupwand.com an YC alumni. The problem is harder than it looks. Now, I am not an expert in PSD format but a PSD that can be automatically converted to HTML has to be specifically formatted. For example, if you merge text layers with image layers, it becomes difficult to extract the information.


We've actually solved this and will be launching in about a week. We've been working on it for over a year now.

It runs inside Photoshop (there is no upload stage nor do you have to open up separate software) and it generates HTML and CSS that looks like a designer wrote it and slices up all your images. It also outputs LESS, SASS, HAML, Slim, Jade and I think there are some other formats I may be forgetting.

Text is output as text. If you use a Google font, it automatically links the fonts in for you.

It does not use absolute positioning so if you modify things like text and it grows taller or shrinks shorter, other items will be positioned properly (initially we did it with absolute positioning but later I figured out an algorithm to make it work the way it should, even with overlapping elements).

It's definitely not an easy problem to solve (hence why we've been working on it for over a year although not full time).

We've done some outputs now and the results are amazing.

I would link you to the website but it looks so bad right now that I don't want to show it. Anyways, we will launch in about a week so look for an announcement. If you have any questions about it, please leave a comment.

Sunny


There are a bunch of those tools which run inside PS, but the point is that I really don't want to run PS. Ever. My designers do and now I have to pay / upgrade licenses too because there is no solution. I think PS is overused and abused for anything 'design' while for programmers it's an annoyance more often than not.


Awesome.We own bunch of CC licenses and I am happy to beta test, if you guys want. :-)


> The problem is harder than it looks.

markupwand.com is pivoting...


isn't that the point?


Thought that GP was not aware of the fact and just linked a relevant company from memory. Makes more sense this way, thanks.


That's true.

Ideally, you'd have to spend some time making sure your .psd is formatted properly instead of just 100 layers names Layer X.


There is certainly the potential for that. The library still has a little ways to go until that's possible.

If anything, I think it would be a cool project to build on top of PSD.rb instead of bloating the core library.


CSS Hat gets you part way there: https://csshat.com/


So, will there be writing .PSD files as well? This would be the über thing. Imagine: upload a .PSD and get back a clean HTML layout + bootstrap_overrides.css


>So, will there be writing .PSD files as well?

Writing PSD files is considerably easier than reading them. To write, you need only support the features you actually use. To read, you must support everything. For example, Photoshop always saves its layers RLE compressed (or it did when I last wrote code to write PSD files, which was about five years ago), but the format supports uncompressed layer data just fine. So if you're just trying to get basic interoperation with Photoshop, you don't have to worry about RLE at all.

>Imagine: upload a .PSD and get back a clean HTML layout + bootstrap_overrides.css

Why would that require writing .PSD files?


Well, so you could iterate on your design - and pick up the changes made in CSS back into the design PSD. And work onwards from there


To be fair, if you wanted to convert a PSD to HTML the last thing you'd need is Bootstrap.


The library supports some writing, but it's mostly limited to translating/scaling layers.


These are great news for developers and an easy way to communicate between designers with developers. Is the beginning of a solution for a big problem.


I'd pay for a Lua port of this. Anyone interested? Drop me a PM.


Truly awesome, cant wait to think of the possibilities with this.


nice. now if someone can port it to C so that everyone can benefit... :)


awesome!


// PSD is not my favourite file format.

I see bro...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: