LinkedIn, and the GDPR age

I should start this post by saying I’m neither a lawyer, nor a GDPR expert.  Possibly both of those facts will become massively apparent in the text that follows.

Also, I’m not a LinkedIn Premium user - so it’s possible I’m missing something obvious by not having access to it.

But anyway, I’ve been thinking about how LinkedIn fits into a GDPR world, and it doesn’t something doesn’t seem quite right to me at the moment.

LinkedIn are in the data business, and they’re very good at protecting that asset.  They tend to be (quite rightly) pro-active in stopping people from extracting data from their systems and pushing it into their own systems.

As such, businesses (recruiters particularly) are encouraged to contact directly within LinkedIn, and they are offered tools to discover people and commence that communication.

Unfortunately, this lack of syncing between LinkedIn and in-house systems can cause a big problem with GDPR.

That is:

What happens if someone says to a recruitment organisation - “Please forget me, and do not contact me again”

In this situation, the organisation is obliged to ‘remove' them from their systems.

At some point in the future another recruiter from the same organisation then finds the person on LinkedIn, without reference to their own systems and messages them using LinkedIn.

What happens next?

By the letter of the law, the organisation may not have done anything wrong.

• The person is no longer in the organisation’s system, they were found on LinkedIn.
• The person was not sent an e-mail, or phoned, they were messaged within LinkedIn.
• The person has consented to have their data held by LinkedIn for the expressed purpose of being contacted by potential recruiters via the platform.

With all this in mind, it may be interpreted that it’s fair game to contact anyone on LinkedIn, regardless of their expressed desire not to be contacted by a particular company.

However, whilst this may be within the definition of the law, it’s pretty clear it’s not in the spirit of the law.

Note - Again I’m not a GDPR expert, nor a lawyer, so can't say for certain that it IS within the definition of the law - nor am I asserting that it is - just that I can imagine that it might be interpreted that way by some people.

And this is where things get complicated for LinkedIn.  I can see a few outcomes of this, but two of them could be extremely worrying for the future on LinkedIn.

Scenario - LinkedIn Premium is seen as an extension of a subscribing organisation’s IT systems.

It could be argued that, whilst LinkedIn is in independent entity, when they provide services to another organisation, their systems then become part of the remit of that subscribing organisation.

I.E. within LinkedIn, any action by a user and the storage of data of that action falls solely within the responsibility of the employer of the user that performs that action.  LinkedIn are not responsible for the use of the data in any way.

On first glance, this looks ideal to LinkedIn - no responsibility!

However, that’s not true - if there’s ever a test case that proves this point, then suddenly LinkedIn becomes a big risk to any organisation that uses it.

Over the course of the last 2 years or so, every data holding organisation in the EU has looked carefully at their data retention and use policies and systems and done what they can to protect themselves - in may cases I’m sure they have changed suppliers and systems since the existing systems have not proven up to scratch in the light of GDPR legislation.

Up to now, I’m not sure that many people have scrutinised LinkedIn in the same way.

At the moment it might be argued that LinkedIn is not supplying the tools to subscribers to allow them to comply with the GDPR legislation.  For example, I’m not aware of any functionality that allows an organisation to state "I wish to completely forget this person, and ensure that I cannot connect, view data on or contact them without their expressed consent”.  If that’s a minimum requirement of any internal system, why would it not be a minimum requirement for LinkedIn?

It could be that once that test case comes, a lot of organisations will take a look at LinkedIn and decide it doesn’t stand up, and it’s no longer worth the risk.

Scenario - LinkedIn, as the data controller, is responsible for the contact made by any users within the system.

This is potentially even worse for LinkedIn.  Since LinkedIn hold the data about people, provide the tools for discovering those people, provide the tools for contacting people, and for relaying those messages, it may be argued that it is up to LinkedIn to provide the mechanism to allow Users to state that they do not wish to be visible to or contacted by a given organisation.

That is, whilst it is another user who is sending the message, it may be that a future test case could state that LinkedIn are responsible for keeping track of who has ‘forgotten’ who.

By not providing that mechanism, and allowing users on the system to make contact when the contact is not welcome and against the target’s wishes, it’s possible that LinkedIn could be argued as being responsible for the unwelcome contact and therefore misuse of data.

Summary

Today, it seems that LinkedIn is in a bit of limbo.

There may be a recognised way to use LinkedIn in the GDPR era - find someone, check in my system that I’m allowed to contact them, go back to LinkedIn and contact them - but in order for that to work it requires the due diligence of recruiters to ensure that the law isn’t broken.

Realistically, something will have to change, or that test case is coming; at some point, someone is going to get an email that is going to break the limbo.

When that happens, I wonder which way it will go..?

Things I still believe in

Over 10 years ago I wrote a blog post on things that I believe in - as a developer, and when I re-read it recently I was amazed at how little has changed.

I'm not sure if that's a good thing, or a bad thing - but it's certainly a thing.

Anyway - here's that list - slightly updated for 2018... it you've seen my talk on Unit Testing recently, you might recognise a few entries.

(opinions are my own, yada yada yada)

• It's easier to re-build a system from its tests than to re-build the tests from their system.


• You can measure code complexity, adherence to standards and test coverage; you can't measure quality of design.


• Formal and flexible are not mutually exclusive.


• The tests should pass, first time, every time (unless you're changing them or the code).


• Test code is production code and it deserves the same level of care.


• Prototypes should always be thrown away.


• Documentation is good, self documenting code is better, code that doesn't need documentation is best.


• If you're getting bogged down in the process then the process is wrong.


• Agility without structure is just hacking.


• Pair programming allows good practices to spread.


• Pair programming allows bad practices to spread.


• Team leaders should be inside the team, not outside it.


• Project Managers are there to facilitate the practice of developing software, not to control it.


• Your customers are not idiots; they always know their business far better than you ever will.


• A long list of referrals for a piece of software does not increase the chances of it being right for you, and shouldn't be considered when evaluating it.


• You can't solve a problem until you know what the problem is. You can't answer a question until the question's been asked.


• Software development is not complex by accident, it's complex by essence.


• Always is never right, and never is always wrong.


• Interesting is not the same as useful.


• Clever is not the same as right.


• The simplest thing that will work is not always the same as the easiest thing that will work.


• It's easier to make readable code correct than it is to make clever code readable.


• If you can't read your tests, then you can't read your documentation.


• There's no better specification document than the customer's voice.


• You can't make your brain bigger, so make your code simpler.


• Sometimes multiple exit points are OK. The same is not true of multiple entry points.


• Collective responsibility means that everyone involved is individually responsible for everything.


• Sometimes it's complex because it needs to be; but you should never be afraid to double check.


• If every time you step forward you get shot down you're fighting for the wrong army.


• If you're always learning you're never bored.


• There are no such things as "Best Practices". Every practice can be improved upon.


• Nothing is exempt from testing. Not even database upgrades or declarative tools.


• It's not enough to collect data, you need to analyse, understand and act upon that data once you have it.


• A long code freeze means a broken process.


• A test hasn't passed until it has failed.


• A test that can't fail isn't a test.


• If you give someone a job, you can't guarantee they'll do it well; If you give someone two jobs you can guarantee they'll do both badly


• Every meeting should start with a statement on its purpose and context, even if everyone in the meeting already knows.

Promises and Lightning Components

In 2015, the ECMA specification included the introduction of Promises, and finally (pun intended) the Javascript world had a way of escaping from callback hell and moving towards a much richer syntax for asynchronous processes.

So, what are promises?

In short, it’s a syntax that allows you to specify callbacks that should execute when a function either ’succeeds’ or ‘fails’ (is resolved, or rejected, in Promise terminology).

For many, they're a way of implementing callbacks in a way that makes a little more sense syntactically, but for others it's a new way of looking at how asynchronous code can be structured that reduces the dependancies between them and provides you with some pretty clever mechanisms.

However, this article isn’t about what promises are, but rather:

How can Promises be used in Lightning Components, and why you would want to?

As with any new feature of Javascript, make sure you double check the browser compatibility to make sure it covers your target brower before implementing anything.

If you want some in depth info on what they are, the best introduction I’ve found is this article on developers.google.com

In addition, Salesforce have provided some very limited documentation on how to use them in Lightning, here.

Whilst the documentations's inclusion can give us hope (Salesforce knows what Promises are and expect them to be used), the documentation itself is pretty slim and doesn’t really go into any depth on when you would use them.

When to use Promises

Promises are the prime candidate for use when executing anything that is asynchronous, and there’s an argument to say that any asynchronous Javascript that you write should return a Promise.

For Lightning Components, the most common example is probably when calling Apex.

The standard pattern for Apex would be something along the lines of:

getData : function( component ) {
    let action = component.get(“c.getData");
        
    action.setCallback(this, function(response) {
            
        let state = response.getState();

        if (state === "SUCCESS") {
            let result = response.getReturnValue();
            // do your success thing
        }
        else if (state === "INCOMPLETE") {
            // do your incomplete thing
        }
        else if (state === "ERROR") {
            // do your error thing
        }
    });
    $A.enqueueAction(action);
}

In order to utilise Promises in a such a function you would:

1. Ensure the function returned a Promise object
2. Call 'resolve' or 'reject' based on whether the function was successful

getData : function( component ) {
    return new Promise( $A.getCallback(
        ( resolve, reject ) => {

        let action = component.get(“c.getData");
    
        action.setCallback(this, function(response) {
            
            let state = response.getState();

            if (state === "SUCCESS") {
                let result = response.getReturnValue();
                // do your success thing
                resolve();
            }
            else if (state === "INCOMPLETE") {
                // do your incomplete thing
                reject();
            }
            else if (state === "ERROR") {
                // do your error thing
                reject();
            }
        });
        $A.enqueueAction(action);
    });
}

You would then call the helper method in the same way as usual

doInit : function( component, event, helper ) {
    helper.getData();
}

So, what are we doing here?

We have updated the helper function so that it now returns a Promise that is constructed with a new function that has two parameters 'resolve' and 'reject'. When the function is called, the Promise is returned and the function that we passed in is immediately executed.

When our function reaches its notional 'success' state (inside the 'state == "SUCCESS" section), we call the 'resolve' function that is passed in.

Similarly, when we get to an error condition, we call 'reject'.

In this simple case, you'll find it hard to see where 'resolve' and 'reject' are defined - because they're not. In this case the Promise will create an empty function for you and the Promise will essentially operate as if it wasn't there at all. The functionality hasn't changed.

Aside - if you're unfamiliar with the 'Arrow Function' notation - E.g. () => { doThing() } - then look here or here. And don't forget to check the browser compatibility.

So the obvious question is.. Why?

What does a Promise give you in such a situation?

Well, if all you are doing it calling a single function that has no dependant children, then nothing. But let's say that you wanted to call "getConfiguration", which called some Apex, and then *only once that was complete* you called "getData".

Without Promises, you'd have 2 obvious solutions:

1. Call "getData" from the 'Success' path of "getConfiguration".
2. Pass "getData" in as a callback on "getConfiguration" and call the callback in the 'Success' path of "getConfiguration"

Neither of these solutions are ideal, though the second is far better than the first.

That is - in the first we introduce an explicit dependancy between getConfiguration and getData. Ideally, this would not be expressed in getConfiguration, but rather in the doInit (or a helper function called by doInit). It is *that* function which decides that the dependancy is important.

The second solution *looks* much better (and is), but it's still not quite right. We now have an extra parameter on getConfiguration for the callback. We *should* also have another callback for the failure path - otherwise we are expressing that only success has a further dependancy, which is a partial leaking of knowledge.

Fulfilling your Promise - resolve and reject

When we introduce Promises, we introduce the notion of 'then'. That is, when we 'call' the Promise, we are able to state that something should happen on 'resolve' (success) or 'reject' (failure), and we do it from *outside* the called function.

Or, to put it another way, 'then' allows us to define the functions 'resolve' and 'reject' that will get passed into our Promise's function when it is constructed.

E.g.

We can pass a single function into 'then', and this will be the 'resolve' function that gets called on success.

doInit : function( component, event, helper ) {
    helper.getConfiguration( component )
      .then( () => { helper.getData( component ) } );
}

Or, if we wanted a failure path that resulted in us calling 'helper.setError', we would pass a second function, which will become the 'reject' function.

doInit : function( component, event, helper ) {
    helper.getConfiguration( component )
      .then( () => { helper.getData( component ) }
           , () => { helper.setError( component ) } );
}

Aside - It's possible that the functions should be wrapped in a call to '$A.getCallback'. You will have seen this in the definition of the Promise above. This is to ensure that any callback is guaranteed to remain within the context of the Lightning Framework, as defined here. I've not witnessed any problem with not including it, although it's worth bearing in mind if you start to get issues on long running operations.

Now, this solution isn't vastly different to passing the two functions directly into the helper function. E.g. like this:

doInit : function( component, event, helper ) {
    helper.getConfiguration( component
                           , () => { helper.getData( component ) }
                           , () => { helper.setError( component ) } );
}

And whilst I might say that I personally don't like the act of passing in the two callbacks directly into the function, personal dislike is probably not a good enough reason to use a new language feature in a business critical system.

So is there a better reason for doing it?

Promising everything, or just something

Thankfully, Promises are more than just a mechanism for callbacks, they are a generic mechanism for *guaranteeing* that 'settled' (fulfilled or rejected) Promises result in a specified behaviour occurring once certain states occur.

When using a simple Promise, we are simply saying that the behaviour should be that the 'resolve' or 'reject' functions get called. But that's not the only option

. For example, we also have:

Promise.all	Will 'resolve' only when *all* the passed in Promises resolve, and will 'reject' if and when *any* of the Promises reject.
Promise.race	Will 'resolve' or 'reject' when the first Promise to respond comes back with a 'resolve' or 'reject'.

Once we add that to the mix, we can do something a little clever...

How about having the component load with a 'loading spinner' that is only switched off when all three calls to Apex respond with success:

doInit : function( component, event, helper ) {
    Promise.all( [ helper.getDataOne( component )
                 , helper.getDataTwo( component )
                 , helper.getDataThree( component ) ] )
           .then( () => { helper.setIsLoaded( component ) } );
}

Or even better - how about we call getConfiguration, then once that’s done we call each of the getData functions, and only when all three of those are finished do we set the flag:

doInit : function( component, event, helper ) {
    helper.getConfiguration( component )
        .then( Promise.all( [ helper.getDataOne( component )
                            , helper.getDataTwo( component )
                            , helper.getDataThree( component ) ] )
                      .then( () => { helper.setIsLoaded( component ) } )
             );
}

Now, just for a second, think about how you would do that without Promises...

Throw it away - Why you shouldn't keep your POC

"Proof of Concepts" are a vital part of many projects, particularly towards the beginning of the project lifecycle, or even in the pre-business case stages.

They are crucial for ensuring that facts are gathered before some particularly risk decisions are made.  Technical or functional, they can address many different concerns and each one can be different, but they all have one thing in common.  They serve to answer questions.

It can be tempting, whilst answering these questions to become attached to the code that you generate.

I would strongly argue that you should almost never keep the code that you build during a POC.  Certainly not to put into a production system.

I'd go so far as to say that planning to keep the code it is often damaging to the proof of concept; planning to throw the code away is liberating, more efficient and makes proof of concepts more effective by focussing the minds on the questions that require answers..

Why do we set out on a proof of concept?

The purpose of a proof of concept is to (by definition):

  * Prove:  Demonstrate the truth or existence of something by evidence or argument.
  * Concept: An idea, a plan or intention.

In most cases, the concept being proven is a technical one.  For example:
  * Will this language be suitable for building x?
  * Can I embed x inside y and get them talking to each other?
  * If I put product x on infrastructure y will it basically stand up?

They can also be functional, but the principles remain the same for both.

It's hard to imagine a proof of concept that cannot be phrased as one or more questions.  In a lot of cases I'd suggest that there's only really one important question with a number of ancillary questions that are used to build a body of evidence.

The implication of embarking on a proof of concept is that when you start you don't know the answer to the questions you're asking.  If you *do* already know the answers, then the POC is of no value to you.

By extension, there is the implication that the questions posed require to be answered as soon as possible in order to support a decision.  If that's not the case then, again, the POC is probably not of value to you.

As such, the only thing that the POC should aim to achieve is to answer the question posed and to do so as quickly as possible.

This is quite different to what we set out to do in our normal software development process. 

We normally know the answer to the main question we're asking (How do we functionally provide a solution to this problem / take advantage of this opportunity), and most of the time is spent focussed on building something that is solid, performs well and generally good enough to live in a production environment - in essence, not answering the question, but producing software.

What process do we follow when embarking on a proof of concept?

Since the aim of a POC is distinct from what we normally set out to achieve, the process for a POC is intrinsically different to that for the development of a production system.

With the main question in mind, you often follow an almost scientific process.  You put forward a hypothesis, you set yourself tasks that are aimed at collecting evidence that will support or deny that hypothesis, you analyse the data, put forward a revised hypothesis and you start again.

You keep going round in this routine until you feel you have an answer to the question and enough evidence to back that answer up.  It is an entirely exploratory process.

Often, you will find that you spend days following avenues that don't lead anywhere, backtrack and reassess, following a zig-zag path through a minefield of wrong answers until you reach the end point.  In this kind of situation, the code you have produced is probably one of the most barnacle riddled messes you have every produced.

But that's OK.  The reason for the POC wasn't to build a codebase, it was to provide an answer to a question and a body of evidence that supports that answer.

To illustrate:

Will this language be suitable for building x?

You may need to check things like that you can build the right type of user interfaces, that APIs can be created, that there are ways of organising code that makes sense for the long term maintenance for the system.

You probably don't need to build a completely functional UI, create a fully functioning API with solid error handling or define the full set of standards for implementing a production quality system in the given language.

That said, if you were building a production system in the language you wouldn't dream of having in incomplete UI, or an API that doesn't handle errors completely or just knocking stuff together in an ad-hoc manner.

Can I embed x inside y and get them talking to each other

You will probably need to define a communication method and prove that it basically works.  Get something up and running that is at least reasonably functional in the "through the middle" test case.

You probably don't need to develop an architecture that is clean with separation of concerns that means the systems are properly independant and backwards compatible with existing integrations. Or that all interactions are properly captured and that exceptional circumstances are dealt with correctly.

That said, if you were building a production system, you'd need to ensure that you define the full layered architecture, understand the implications of lost messages, prove the level of chat that will occur between the systems.  On top of that you need to know that you don't impact pre-existing behaviour or APIs.

If I put product x on infrastructure y will it basically stand up?

You probably need to just get the software on there and run your automated tests.  Maybe you need to prove the performance and so you'll put together some ad-hoc performance scripts.

You probably don't need to prove that your release mechanism is solid and repeatable, or ensure that your automated tests cover some of the peculiarities of the new infrastructure, or that you have a good set of long term performance test scripts that drop into your standard development and deployment process.

That said, if you were building a production system, you'd need to know exactly how the deployments worked, fit it into your existing continuous delivery suite, performance test and analyse on an automated schedule.

Production development and Proof of Concept development is not the same

The point is, when you are building a production system you have to do a lot of leg-work; you know you can validate all the input being submitted in a form, or coming through an API - you just have to do it.

You need to ensure that the functionality you're providing works in the majority of use-cases, and if you're working in a TDD environment then you will prove that by writing automated tests before you've even started creating that functionality.

When you're building a proof of concept, not only should these tests be a lower priority, I would argue that they should be *no priority whatsoever*, unless they serve to test the concept that you're trying to prove.

That is,  you're not usually trying to ensure that this piece of code works in all use-cases, but rather that this concept works in the general case with a degree of certainty that you can *extend* it to all cases.

Ultimately, the important deliverable of a POC is proof that the concept works, or doesn't work; the exploration of ideas and the conclusion you come to; the journey of discovery and the destination of the answer to the question originally posed.

That is intellectual currency, not software.  The important delivery of a production build is the software that is built.

That is the fundamental difference, and why you should throw your code away.

The opportunity cost of delaying software releases

Let me paint a simple picture (but with lots of numbers).

Some software has been built.  It generates revenue (or reduces cost) associated with sales, but the effect is not immediate.  It could be the implementation of a process change that takes a little time to bed in, or the release of a new optional extra that not everyone will want immediately.

It is expected that when it is initially released there’ll be a small effect.  Over the next 6 months there will be an accelerating uptake until it reaches saturation point and levels off.

Nothing particularly unusual about that plan.  It probably describes a lot of small scale software projects.
Now let’s put some numbers against that.

At saturation point it’s expected to generate / save an amount equal to 2% of the total revenue of the business.  It might be an ambitious number, but it’s not unrealistic.

The business initially generates £250k a month, and experiences steady growth of around 10% a year.

What does the revenue generation of that software look like over the first 12 months?

It’s pretty easy to calculate, plugging in some percentages that reflect the uptake curve:

Period	Original Business Revenue	Software Revenue Generation	Additional Revenue
1	£250,000.00	0.2%	£500.00
2	£252,500.00	0.5%	£1,262.50
3	£255,025.00	1.1%	£2,805.28
4	£257,575.25	1.6%	£4,121.20
5	£260,151.00	1.9%	£4,942.87
6	£262,752.51	2.0%	£5,255.05
7	£265,380.04	2.0%	£5,307.60
8	£268,033.84	2.0%	£5,360.68
9	£270,714.18	2.0%	£5,414.28
10	£273,421.32	2.0%	£5,468.43
11	£276,155.53	2.0%	£5,523.11
12	£278,917.09	2.0%	£5,578.34
		Total:	£51,539.34

Or, shown on a graph:

So, here’s a question:

What is the opportunity cost of delaying the release by 2 months?

The initial thought might be that the effect isn’t that significant, as the software doesn’t generate a huge amount of cash in the first couple of months.

Modelling it, we end up with this:

Period	Original Business Revenue	Software Revenue Generation	Additional Revenue
1	£250,000.00		£-
2	£252,500.00		£-
3	£255,025.00	0.2%	£510.05
4	£257,575.25	0.5%	£1,287.88
5	£260,151.00	1.1%	£2,861.66
6	£262,752.51	1.6%	£4,204.04
7	£265,380.04	1.9%	£5,042.22
8	£268,033.84	2.0%	£5,360.68
9	£270,714.18	2.0%	£5,414.28
10	£273,421.32	2.0%	£5,468.43
11	£276,155.53	2.0%	£5,523.11
12	£278,917.09	2.0%	£5,578.34
		Total:	£41,250.69

Let’s show that on a comparative graph, showing monthly generated revenue:

Or, even more illustrative, the total generated revenue:

By releasing 2 months later, we do not lose the first 2 months revenue – we lose the revenue roughly equivalent to P5 and P6.

Why?

When we release in P3, we don’t immediately get the P3 revenue we would have got.  Instead we get something roughly equivalent to P1 (it’s slightly higher because the business generates a little more revenue overall in P3 than it did in P1).

This trend continues in P3 through to P8, where the late release finally reaches saturation point (2 periods later than the early release – of course).

Throughout the whole of P1 to P7 the late release has an opportunity cost associated.  That opportunity cost is never recovered later in the software’s lifespan as the revenue / cost we could have generated the effect from is gone.

If the business was not growing, this would amount to a total equal to the last 2 periods of the year.

In our specific example, the total cost of delaying the release for 2 months amounts to 20% of the original expected revenue generation for the software project in the first year.

And this opportunity cost is solely related to the way in which the revenue will be generated; the rate at which the uptake comes in over the first 6 months.

Or to put it another way – in this example, if you were to increase or decrease the revenue of the business or the percentage generation at which you reach saturation point the cost will always be 20%.

So, when you’re thinking of delaying the release of software it’s probably worth taking a look, modelling your expected uptake and revenue generation to calculate just how much that will cost you…