

A better way to buy code. Use tests to buy code - cashmonkey85
https://codeswap.io/

======
ColinWright
This is a lovely idea, but I'd like to share with readers an experience I had
a while ago.

I was looking to outsource some coding, so I advertised and chose someone that
seemed reasonable. I asked for some sample code doing similar things to what I
wanted, they responded with some snippets that looked right, so we entered an
agreement. I would provide some sample input and output, they would provide
code that produced the right results. Once I verified that it did what was
required, I would pay.

It was a problem domain that wasn't overly complex, and I thought it would be
an ideal task to outsource. It was a lot more complex that the example I'll
give here, but I want to give you a flavour of how it all went wrong. This is
_not_ the task, just something to explain _how_ it went wrong.

I asked for code that would return "True" if the 2nd and 4th characters in a
supplied string were the same, but otherwise all characters were different. So
here are some examples of input and output:

    
    
        severity    True
        vagabonds   True
        unknowable  True
        unsnarled   True
        telegraphs  True
        robot       True
    
        faunas      False
        flimsily    False
        freighting  False
        rummaging   False
        mosaic      False
        shushes     False
        conflicting False
    

Remember, the actual task was described in full, was much more complex than
this, and we were simply providing some example test cases.

What we got back was the equivalent of this:

    
    
        def test_string(x):
    
            if x == 'severity'    return True
            if x == 'vagabonds'   return True
            if x == 'unknowable'  return True
            if x == 'unsnarled'   return True
            if x == 'telegraphs'  return True
            if x == 'robot'       return True
    
            if x == 'faunas'      return False
            if x == 'flimsily'    return False
            if x == 'freighting'  return False
            if x == 'rummaging'   return False
            if x == 'mosaic'      return False
            if x == 'shushes'     return False
            if x == 'conflicting' return False
    

So to anyone who tries to specify their coding tasks by using tests - beware.
There are coders who will strive to meet the exact letter of the requirements,
and then demand payment.

For what it's worth, we never resolved this and eventually paid an agreed
amount to make them go away. We took this as a reasonably expensive lesson
about some of the potential pitfalls of out-sourcing.

~~~
cashmonkey85
Hey, thanks for the example. This is definitely something we want to avoid
with this platform and have thought of ways to mitigate it.

One way is we will allow the buyer to further test submissions as black boxes
to see if the output matches their expectations.

The second way is with a rating system as extra information for the buyer and
an incentive for the coders to write good code.

~~~
ColinWright
How do you avoid this scenario:

    
    
      * Customer creates job spec
      * Customer creates white examples
      * Customer creates black examples
      * Coder bids
      * Customer accepts
      * Customer sends spec
      * Customer sends white examples
      * Coder delivers code
      * Code fails black examples
      * Customer refuses to pay
      * Coder claims code meets spec
    

??

~~~
codeswap
The platform is weighted in favour of the buyers. What I mean by that is the
coders compete to win the payout. The buyer doesn't see or own the source code
until they pay. Many coders can submit solutions and the buyer can pick the
best one to purchase. It's up to the buyer to query the submission fully as a
black box and to trust the rating system.

