I was actually thinking about doing a write up on the issues I've had but this seems to make me think I should do so AFTER I find someplace else to go. Right now GitHub is the likely destination but would love to hear other suggestions.
(edit: looks like HN is severely limiting my reply rate so apologies for delays)
We're trying to focus on frustrating pages/experiences rather than number of network calls and such, because while the latter is correlated, the former (frustrating slow experiences) is really the bar and the higher priority.
In terms of the ToS I'm not from legal so can't say (still looking into it), but have definitely had conversations with users on public forums about performance issues, and afaik no one has been accused of violating their ToS.
(edit: since I can't reply due to HN limits I'll try to add some stuff in this edit)
"target things that are easier to fix than those with highest impact" -> this is a good point and something we're trying to do. Engineers know the former (easier to fix) pretty readily, but identifying "highest impact" requires some work, so I'm (as a PM) always trying to find out. It's of course some combination of these two (low hanging fruit, high impact items) that forms the priority list.
------ @igetspam (moved followup into a reply to trigger notification)
"perf to take over company for 6mo-1yr" I'm not in the position to make that level of decisions, but can certainly pass the feedback up the chain. The perf team is trying their best though, so any info anyone can provide us can help us apply our resources in the right place
On a cloud-based classic software project (which has less than 200 issues) opening a link to an issue it takes 4.8 seconds for the page to complete rendering and the progress bar at the top of the screen to disappear.
Opening a Kanban board with 11 issues displayed? 4.2 seconds for the page to load.
Click an issue on the board? 2.5 seconds for the details to pop up.
Close that task details modal - literally just closing a window? 200 milliseconds. Not to load a page - just to close a modal!
In case I'm being hard on cloud Jira by insisting on using a classic project, I also checked with a 'Next-gen software project' with less than 2000 issues.
I click a link to view a particular comment on an issue. 4.8 seconds until the issue, comment and buttons have all loaded.
I choose to view a board? 9.9 seconds from entering the URL to the page load completing.
I'm viewing the board and I want to view a single issue's page. I click the issue and the details modal pops up - and just as I click on the link to the details, the link moves because the epic details have loaded, and been put to the left of the link I was going for, causing me to click the wrong thing. So this slow loading is a nontrivial usability problem.
View a single issue, then click the projects dropdown menu. The time, to display a drop-down menu with three items? 200 milliseconds.
This is what people mean when they say the performance problems are everywhere - viewing issues, viewing boards, viewing comments, opening dropdowns, closing modals? It's all slow.
And if you imagine a backlog grooming meeting that involves a lot of switching back and forth between pages and updating tickets? You get to wait through a great many of these several-second pageloads.
Oracle comes to mind.
Maybe this person is writing a fiction story where the protagonist is using Jira and they are detailing how they spend their day.
Its like a John Steinbeck novel
No, this is a quote from the comment:
"This is what people mean when they say the performance problems are everywhere - viewing issues, viewing boards, viewing comments, opening dropdowns, closing modals? It's all slow."
"Atlassian Cloud ToS section 3.3(I) prohibits discussing performance issues"
No sane judge would agree with your interpretation.
Thank you for the numbers -> I agree these are slow, and I can guarantee you that the Jira team is working on it (though I can't talk about details). These numbers are definitely outside of the goals.
I appreciate the call out of "page to complete rendering and the progress bar at the top of the screen to disappear" and "until the issue, comment and buttons have all loaded". In a dream world of course, everything would load in < 1s (everything drawn, everything interactive), but working our way down to that will take time.
We're currently looking at each use case to understand the '(a) paint faster vs (b) interactive faster' tradeoff and trying to decide which cases the user has a better experience with (a) or (b). In Confluence this is clearer in some places than in others, but in Jira it's less clear I think (I work on Confluence, I probably shouldn't speak for Jira specifics).
It always comes down to a limitation of resources though, which is why we're always hoping to get as specific feedback as possible.
It's important you understand that "everything loading in <1s" would still be unacceptably slow - that is still an order of magnitude too slow.
That is not "a dream world" - not even close. A well built tool like this, meeting standard expectations (i.e. table stakes), would hit <50ms for the end user - the vast majority of the time. A "dream world" would be more like 10ms.
You should be targeting <200ms for 99% of user-facing interactions. That is the baseline standard/minimum expected.
This is why people are saying the company needs to make a major shift on this - you're not just out of the ballpark of table stakes here, you're barely in the same county!
It cannot be overstated how far off the mark you are here. There's a fundamental missetting of expectations and understanding of what is acceptable.
I just tested a HN profile page (famously one of the lightest weight non-static websites) and it takes between 300ms and 600ms to load. I'm not saying that Jira can't improve, but if HN isn't hitting 250ms then I think telling the Jira guys that nothing less than <200ms is the minimum standard is unrealistic.
Jira is also much more interactive than HN. You are sitting 10+ people in a room with some half asleep scrum master opening the wrong issue, have to go back and open the correct one, search again for some related issue you though was fixed last month. Refresh the board to make sure you didnt forget to fill in one field so it ends up in the wrong column, etc etc.
1 sec per click in a situation like this is a joke, and that's just their goal. Reality is 4sec+ as OP mentioned, often even more.
A 200ms page load is incredibly fast.
Still, I tested your profile page on Google PageSpeed and it came out at a 300ms load time.
When taking that kind of approach, you don't have to wait for the slowest thing to come in - eg with a normal BE render, you might need to pull up the user's profile and settings, A/B testing flags, the current footer config or whatever.
eg if you're on the page for viewing a single ticket, you can request the ticket data immediately, and render it as soon as it's available - even if other parts of the page aren't finished yet. True it may be more like 200-300ms to have the entire thing be 100% complete, but all parts of the page are not of equal importance and holding up the main content while loading the rest isn't necessary.
If you are doing a full BE render, it's still totally possible to hit that 100ms mark, but indeed dramatically more difficult.
You're right, I apologize for not being clear. We're targeting 1s for "Initial loads" on new tabs/new navigation, which I assume you're referring to. Our target for 'transitions' is different.
If however the numbers you're referring to are "initial load" numbers, then I'm not sure.
(edit: and action responses again are also a separate category. Our largest number of complaints are about 'page load times' in Confluence, so most conversations center around that)
But Jira currently is so slow that 1s would be a great improvement. I am using it at work and regret it, unfortunately.
New load, you should really be hitting 200ms as your 95th percentile - 300ms or so would be decent still. "Transitions" should hit 100ms 95th, 150ms would be decent.
If you did hit 100ms across the board, you'd be rewarded by your customers/users psychologically considering the interactions as being effectively instantaneous. So it really is worth setting a super high bar as your target here (esp given you need a bit of breathing room for future changes too).
Most of what we've seen online are nowhere near this level of detail (X-ms for Y-%ile for Z-type of load)
(edit: clarified request)
On what users experience as effectively "instantaneous", that's from experience on UX engineering and industry standards - https://www.nngroup.com/articles/response-times-3-important-...
On the other noted times, they're just a general range of what can be expected from a reasonably well-built tool of this nature. Obviously much simpler systems should be drastically faster, but project management tools do tend to be processing quite a bit of data and so do involve _some_ amount of inherent "weight", but that isn't an excuse for very poor perf.
That said, I imagine if your PMs do some research and go ahead and try using some of the common project management tools, you should get a good idea. ;) Keep in mind speeds to Australia (assuming Atlassian is operated mostly there?) will likely show them in a much worse light than typical perf experienced in the US/UK/EU areas.
The time to first load is derived from the fact that you're running essentially the equivalent of many "transition" type interactions, but they should be run almost entirely in parallel, so roughly 2x between "transition" and "new load" is a reasonable allowance.
However I've not seen guidance on whether these should be P90 or P95 or P99 measures for example though. We've selected something internally, but obviously selecting amongst three 'measurement points' could drastically change general user's experience.
(HN is throttling my replies so apologies for delay)
A big part is simply how far you are in your journey of getting good at performance - if your p50 is still garbage, there's not much point in focussing on your p99 measurements. You should be targeting the p99 long term, but focus on the p50/p90 for now.
It's super important to target and make long term decisions around the p99 though, because, e.g., making a 100x improvement is not possible through little iterative changes over 2-3 years. You need a base to work from where that 100x is fundamentally achievable, which requires thinking from first principles and slightly getting out of the typical product mindset.
I also find the typical product mindset tends to result in focussing a lot on the "this quarter/next quarter" goals, but neglecting the "8/12 quarters from now" as a result.
Beyond short term/long term goals, the choice is largely just down to what the product is/does. Even ignoring all current architectural choices, there are some fundamentals where certain things must always be faster/slower - e.g. sync writes will typically be a fair bit slower than reads, and typically occur much less often, complex dynamic queries which can't be pre-optimised require DB scanning but are much less common.
For these kinds of tools, where most of the interaction is reads, mostly on predefined or predefined + small extra filtering, and reading/writing on individual resources (ie tickets), you can get p99 numbers trending towards the 100ms mark eventually - there's very little which truly can't get to that level with clever enough engineering.
Of course I imagine Google tends to be looking more at their p99.9/p99.99/pmax/etc(!), at least for their absolute highest volume systems.
None of us are going to be getting to that point, but it's often worth thinking about engineering principles against a super high bar - it often helps people to open their minds a bit more and think more outside the box when given a really dramatic goal magnitudes beyond their existing mindset.
Of course you're not expecting to really get to that level, but anchoring that way can achieve amazing things. I've done that with a lot of success at my company and we actually did manage to achieve a few originally thought to be totally unrealistic.
Sorry if that’s pointed, but it’s sort of meant to be incredulous (but hopefully not offensive).
I would never encourage anyone to violate a ToS of another product and apologize for anyone that was considering doing it due to my ask.
I think these are other possibilities:
1) (as stated) other products don't have such ToS
2) other products may have published their own metrics and made them available for consumption
3) from a more legal in depth standpoint, maybe other companies have such ToS terms but have clarified them to some point that makes them more clear about when they apply and when they don't
I don't know what the overlap is between Atlassian users and IT admins though - my previous job was on the vSphere UI and if you happen to know about the death of the Flash based client, this is not too far off.
Hopefully users stay willing to engage with us so we can improve the product as fast as possible.
If it means something else then I'm not aware if we do it or not.
Unacceptable to who, you have a faster provider for cheaper, with as many features???
Im pretty sure he doesnt because if he could he would go there. There are tradeoffs and Atlassian has many project they are working on. They understand that there is room for improvement in performance. Its one of Atlassian's priorities, it is a tech company (a pretty good one I would say).
I guess one question is about server redundancy. Where is this guy loading from and where is the server he is loading from? Getting things below 1s is nearing the speed of the connection itself. Also at that speed there is deminishing returns. Something that happens at 1s vs .5s doesnt make you twice as fast when you dont even have the response time to move your mouse and click on the next item in .5s.
Sometimes techies just love to argue. You are doing great Atlassian and have tons of features. But maybe it is time to revisit and refactor some of your older tools.
> Getting things below 1s is nearing the speed of the connection itself
That is absolutely false. Internet latency is actually very low - even e.g. Paris to NZ is only about 270ms RTT, and you _do not_ need multiple full round trips to the application server for an encrypted connection - on the modern internet, connections are held open, and initial TLS termination is done at local PoPs.
For services like this - as they are sharded with customer tenancy - are usually located at least in the same vague area as the customer (e.g. within North America, Western Europe, APAC etc).
For most users of things like Atlassian products, that typically results in a base networking latency of <30ms, often even <10ms in good conditions.
Really well engineered products can even operate in multiple regions at once - offering that sort of latency globally.
> Im pretty sure he doesnt because if he could he would go there
Yeah, we don't use any Atlassian products - partly for this reason. We use many Atlassian-comparable tools which have the featureset we want and which are drastically faster.
> when you dont even have the response time to move your mouse and click on the next item in .5s.
There is clear documented understanding of how UX is affected by with various levels of latency - https://www.nngroup.com/articles/response-times-3-important-...
> Sometimes techies just love to argue
Not really, I have no particular investment in this - I don't use any Atlassian product, nor do I plan to even if they make massive perf improvements.
But I do have an objective grasp - for tools like this - of what's possible, what good looks like, and what user expectations look like.
> no dont give into this guy
I don't expect Atlassian is going to make any major decisions entirely based on my feedback here, but it is useful data/input for exploration, and I do feel it's right to point out that they're looking in the wrong ballpark when it comes to the scale of improvement needed.
It's the network latency equivalent of a million kilometres of fibre!
In my mind, anyone doing UI development and seeing user interactions taking over 1 second should be asking themselves "did the user just try to operate on more than 10^6 of something?" and if the answer is no, start operating under the assumption that they've made a mistake.
A gateway solely adds 50 ms. So I'm not really sure where you get your numbers/benchmarks from... They are unrealistic
Ocelot would be a better example of an gateway https://github.com/ThreeMammals/Ocelot
Used for scaling up web traffic or creating bff's ( backends for frontends)
I personally would still say 50ms is super, super slow for an application gateway - a well designed one using e.g. nginx/openresty, lambda@edge, or simply writing another application server etc can easily do that job with an addition of <0.1ms processing time (assuming no additional network calls or heavy work), and maybe 0.3ms for additional connection establishment if it hasn't been optimised to use persistent connections.
If it is e.g. making a DB request to check auth, I would highlight that this _is_ backend processing time, not inherent or unoptimisable overhead. e.g. it's totally feasible to do auth checks without making any async calls, just need a bit of crypto and to allocate some memory for tracking revoked tokens - does add a bit of complexity, but likely worth it for the super hot path.
BFFs would not really need to add anything beyond ~1ms or so, but you do hit the lowest common denominator - in that you have to wait for the slowest thing to complete, even if everything is happening in parallel.
BFFs definitely benefit in simplifying client-side code, but at the downside of increased overall latency and potentially resilience which could be achieved by decoupling unrelated components.
As such, I wouldn't expect the Atlassian products to use BFF patterns - for them it's better to throw 1k requests down a single HTTP 2/3 connection and render each part of the page when it's available. I have heard their FEs are very complex, which I think would probably support that assessment.
Gateways can add a lot of functionality. Even Graphql can be used as a gateway.
It's not all "dumb forwarding" and I would be very surprised that you find any sub ms benchmarks.
Amazon has a one million dollar award if you get the page to load under 10 ms. So that's what you are expecting by default on a saas in your previous comment.
It's still unrealistic.
e.g. Envoy (which replaced Ocelot in Microsoft's .net microservice reference architecture) has a significantly lower latency cost (1)
Your reference to an Amazon reward is interesting as it's quite easy to get pages to load under 10ms in the right conditions. Perhaps you can provide a link to more information?
If you want to really focus on performance, you can choose not to use anything like that off the shelf and do it all in a fraction of a millisecond. It actually isn't difficult - you just need to not get stuck into a dependency on something heavy.
For my company's backend, our entire middleware stack incl auth checks is around the 1-2ms level including hitting a DB server to check for token revocation. That's all there is between the end user and our application code, plus network latency. We didn't do anything particularly clever or special. But we didn't use any frameworks or heavy magic products - just Go's net/http, the chi router and Lambda@Edge.
> In a dream world of course, everything would load in < 1s (everything drawn, everything interactive),
As a contractor, I have more or less walked out of or refused interviews on discovering Atlassian toolset was in use. It's not because I hate your tooling (it is visually nice and very featureful), it's because the culture that delivered this software is antithetical to anything I look for in a software project I want to use or contribute to. How can I possibly do my job to any degree of satisfaction when I'm tracking work in a tool that requires 15 seconds between mouse clicks? That is the reality of Jira, and as a result I refuse to use it, or work for people who find that acceptable, because it's a "broken window" that tells me much more about the target environment than merely something about suboptimal bug trackers.
Your page budget should be 100ms max, given all your tools actually do are track a couple of text fields in a pleasing style. Whoever the architecture astronauts are at Atlassian that created the current mess, flush them out, no seat is too senior -- this is an existential issue for your business.
All these systems suck. You learn to live with them, for me I do this:
Everything goes in OmniFocus, I have a keyboard shortcut to create a task that takes <1sec, hit enter twice and it's stored. Twice a day I go though all the tasks I entered this way, and I either mark them done or assign them to various projects/tags/labels I have setup on OmniFocus.
15 mins before I finish work for the day at a client, I update whatever ticket system they use (mostly Jira, but also sometimes even worse things like servicenow) and also whatever enterprise crapware my agency uses (usually some sap based bollocks).
The last 15 mins suck. But it's part of the deal. I can't imagine how strongly you feel to turn down contractor rates due to a ticket system.. I mean, come on?
Edit: Also - btw -- if you're on a Mac the app-store 'fat-app' version of Jira is about 10x better than using the web interface, I suggest you give it a try.
Jira is the mouldy 80386, and the client's culture is that basement where such things belong. I can't see how this is even being precious. I can find solid work on good teams with smart people anywhere, there is no reason I need to work in a basement permanently damaging my lungs.
Lame analogy, I know, but it's close enough.
> I can't imagine how strongly you feel to turn down contractor rates due to a ticket system.. I mean, come on?
It may be the case that they are in such high demand they have practically free choice of work. That's how I interpreted it, at least.
Yeah although it doesn't exactly help in figuring out how to resolve, I think this can be a good grounding in what the product fundamentals actually are and figuring out which over-engineering of those fundamentals is translating into speed problems
I often feel that product people view this type of problem in the wrong way - when you're starting at 5-10s, little incremental A/B tested tweaks are not going to get you down to 50-100ms. A 100x diff requires you to rethink from first principles - it's impossible to get there otherwise
Of course this is also why incumbents get disrupted by startups!
It says they're so afraid of the quality of their product they'd rather litigate their customers than fix their product.
Certainly if someone asked me what I know about Atlassian, this would now be one of the first things that come to mind.
FWIW our on-prem install uses <1s for opening issues, running a search etc. Too bad that's a dream world you've decided should no longer be...
We’re talking 3-5 redirects for some things they could just proxy on their backend. It’s dumb and there’s no amount of hardware or bandwidth a client can throw at the problem to fix it.
I mean, I don't do web stuff (yet) but I can't imagine it's that difficult to figure out where several seconds get spent.
I’m only sort of joking here, since it can be weirdly difficult to actually just do the job you’re being paid to do in some orgs. I once got paid handsomely to deliver almost nothing for six months since all the layers above me were busy either talking back and forth or not even caring at all. It bored me so much that I had to leave, but the money was great.
There weren’t even any disappointed customers because they just allocated budget and forgot about the project. It wasn’t their own money they were spending, after all.
Being an engineer is weird sometimes.
It is only a tradeoff if you're at the Pareto optimality frontier  for those two things.
I seriously doubt that you are. You should absolutely be able to have more of both.
I would recommend to you personally two things: Open the debugger, and load a page with an issue on it in any environment. Look at the timeline of incoming resources, not just for how long the total takes but also all the other times. You will learn a lot if you haven't done this yet. It will be much more informative than anything we can tell you.
Second, once an issue is loaded, right click on almost anything in the page (description, title, whatever) and select "Inspect Element". Look at how many layers deep you are in the HTML.
I also find it useful to Save the Web Page (Complete) once it's all done rendering, then load it from disk with the network tab loaded in the debugger. It can give a quick & dirty read on how much time it takes just to render the page, separate from all network and server-side code issues.
I have a bit of a pet theory that a lot of modern slowdown on the web is simply how much of the web is literally dozens and dozens of DOM layers deep in containers that are all technically resizeable (even though it is always going to have one element in it, or could be fixed in some other simple way), so the browser layout engine is stressed to the limit because of all the nested O(n) & O(n log n) stuff going on. (It must not be a true O(n^2) because our pages would never load at all, but even the well-optimized browser engines can just be drowned in nodes.) I don't have enough front-end experience to be sure, but both times I took a static snapshot of a page for some local UI I had access to that was straight-up 2+ seconds to render from disk, I was able to go in and just start slicing away at the tags to get a page that was virtually identical (not quite, but close enough) that rendered in a small fraction of a second, just with HTML changes.
My guess is that fixing the network issues will be a nightmare, because the 5 Whys analysis probably lands you at Conway's Law around #4 or #5. But, assuming you also have a client-side rendering issue (I don't use JIRA Cloud (yet) but I can vouch that the server product does), you may be able to get some traction just by peeling away an engineer to take a snapshot of the page and see what it takes to produce a page that looks (nearly) identical but renders more quickly. That will not itself be a "solution" but it'll probably provide a lot of insight.
We have looked into the network issues and some of it is similar to what you stated, we do have a known minimum given our chosen cloud infrastructure (separate from our software performance) - we obviously recognize we're not at that limit yet either though.
I have not tried what you mentioned above (load from disk), but I will give it a shot -> it may also give us a clue on how to make our performance testing lower variance, come to think of it......
(apologies for slow replies, HN is still throttling me due to downvotes)
When faced with long-tail performance problems, it's often better to target the things which are easier to fix rather than the highest impact fixes. Making 20 relatively low-impact things faster can easily be better than improving 10 individually high impact things.
I'm actually going to look into shorting Atlassian now.
a) the public facing jira.atlassian.com is Server/DC instance - but this instance is only used for customer/outside world facing tickets (I think)
b) internally for our own development we use a few (several?) cloud instances -> but I can only speak to those I interact with (Conf Cloud and Jira Cloud primarily).
It's not really a problem with a certain page or a certain action: it's a systemic issue, that can only be solved with a systemic change.
This has come up before here in HN . From my point of view, ignoring the issue around number of calls/performance and all feedback regarding it is the root cause for the slowness.
Thank you for reiterating this point, and I'll try to shed some light on this. We actually are working on systemic changes to try to make this lighter/better but I can't talk about specifics until the feature is available.
On the other hand, any level of specificity is great, for example:
1) full page loads are slower and more annoying than Transitions (or, vice versa)
2) loading Home page is slower and more annoying than Search Results (or, vice versa)
3) waiting for the editor to load is more annoying than X/Y/Z
Even systemic changes require individual work for applying to these different views, so any level of specific feedback would be helpful.
(also it looks like HN is limiting my reply rate so apologies for any slowness)
I also use Confluence and JIRA regularly, and can confirm that they are the slowest most terrible software that I use on a regular basis. Every single page load and transition is slow and terrible.
Asking "which one is the highest priority" is like asking which body part I'd least prefer you amputate. The answer is: please don't amputate any of them.
It's as if I asked you to dig out a hole for pouring foundation for a house. The answer to "which shovelfull of dirt has the highest priority" is all of them. Just start shoveling. It's not done until you've dug the entire hole.
It's like the exterminator asking which specific cockroach is bothering me the most. (It's Andy. Andy the cockroach is the most annoying one, so please deal with her first).
What I, and many many other commenters, are trying to tell you is that the entire product is slow and terrible (not your fault. I'm guessing you're new and just trying to improve things, and I hope you succeed!). If it were a building, I'd call it a teardown. If it were a car, I'd call it totaled.
It doesn't matter what page or interaction you start with. Just start shoveling.
Thanks for the understanding! Indeed I haven't been at Atlassian that long, but that's not a good excuse: it's my problem to own.
I appreciate the reinforcement of "fix everything", and I assure you we're trying our best to do so. As a PM it is my natural instinct (and literal job) to prioritize, so I'm always looking for more details to do so.
I can understand that my request for details can imply that I'm either not listening or not believing the feedback, but that is not the case -> I do understand everything is slow and needs fixing.
We are actively looking for other solutions outside of Atlassian, specifically because the demands to switch to your cloud offerings. We simply do not trust your cloud.
We also have a higher compliance requirement, since we can have potential snippets of production data. Our Jira/Confluence systems are highly isolated inside a high compliance solution. We can verify and prove that these machines do not leak.
The Atlassian cloud is completely unacceptable in every way possible. And going from $1200ish year to $20000 per year with data center is laughably horrendous - for the same exact features.
Unless Atlassian changes its direction, your software is that of the walking dead. We have a absolute hard timelimit of 2024, but in reality, 2022. We'd like to still use it and pay you appropriately, but we're not about to compromise our data security handling procedures so you can funnel more people into a cloud service... And judging by the comments here, is pretty damn terrible.
* define a metric
* measure it automatically with every commit
* define a success threshold
* make changes to get yourself under the threshold
* prohibit further changes which bring you above the threshold
Just do it like that for pretty much every view in the system.
question a) My understanding is that performance numbers fluctuate a LOT, even at sampling in the tens of thousands. Do you have any recommendations of tools or methods to reduce this variance?
comment b) we're definitely trying to do this but we're not there yet - most of our metrics don't meet goals we set. Instead the blocking goals must be 'don't make it any worse', which is doable -> but it doesn't necessarily make anything better yet (thus all the questions about what is most annoying that we can fix first).
Hopefully point (b) is clear - I'm not saying "our performance is great/good/acceptable", just the best I can do (as a PM) is try to figure out what to prioritize to fix.
The high variance does give you two tactical problems. First, how do you keep performance from getting worse? Typically you would set a threshold on the metrics, and prevent checking in code that breaks the threshold. With high variance you clearly cannot do this. Instead, make the barrier soft. If the performance tests break the threshold, then you need to get signoff from a manager or senior engineer. This way, you can continue to make coding progress while adding just enough friction that people are careful about making performance worse.
The second problem of high variance is showing that you're making progress. However, for you, this isn't a real problem. You're not talking about cutting 500 microseconds off a 16 millisecond frame render. You need to cut 5-25 second page loads down by a factor of 10 at least. There must be dozens of dead obvious problems taking up seconds of run time. Is Confluence's performance so atrocious that you couldn't statistically measure cutting the page load time in half?
Showing that we're making progress isn't as much of a problem - similar to what you stated, the fixes themselves target large enough value that it's measurable at volume for sure, and even in testing.
The main issue is "degradations" -> catching any check-ins that can degrade performance. These are usually small individually (lets say, low double digit MS) within the variance noise), but add up over time, and by the time the degradation is really measurable, its complicated tracking down the root cause. Hopefully I described that in a way that makes sense?
Any suggestions welcome.
(Edit: downvoted too much and replies are throttled again)
Thanks for the detail! will definitely take this to eng team for process discussion.
In your PR, you link to the tool showing the performance diff of your PR. The tool shows the absolute and relative differences of performance from the base version of code. It also tracks the variance of each metric over time, so it can kind of guess which metrics have degraded, though this doesn't work consistently. The tool tries to highlight the likely degraded metrics so the engineer can better understand what went wrong.
If the metrics are better, great! Merge it! If they are worse, the key is to discuss them (quickly in Slack), and decide if they are just from the variance, a necessary performance degradation, or a problem in the code. Typically it's straightforward: the decreased metrics either are unrelated to the change or they are worth looking into.
The key here is not to make the system too rigid. Good code changes cannot be slowed down. Performance issues need to be caught. The approvers need to be fast, and to mostly trust the engineers to care enough to notice and fix the issues themselves.
We also check the performance diffs weekly to catch hidden regressions.
IF YOUR ORGANIZATION DOES NOT VALUE AND REWARD PERFORMANCE IMPROVEMENTS, NONE OF THIS WILL WORK. Your engineers will see the real incentive system, and resist performance improvements. Personally, I don't believe that Atlassian cares at all about performance, otherwise it never would have gotten this bad. Engineers love making things faster, and if they've stopped optimizing performance it's usually because the company discourages it.
Instead they “prioritized” a legal change in the ToS.
1) markdown macro (limits markdown to the body content, and not interacting with other macros or styling)
2) copy/paste markdown -> autoconvert to WYSIWYG (limits markdown to copy/pasting, so no editing Markdown inside)
3) "markdown pages" (something like 'this page is Markdown only, no WYSIWYG)
I make no comments/promises on any of these becoming real but looking for what's most valuable
1. Don't be Slack / Teams :)
What I mean by this is if you support it, support it correctly rather than "markdown as keyboard shortcuts" that some products do where if you make the simplest of editing changes, your "markdown" isn't recognized.
2. Devs want to consistently use Markdown across their entire suite
At my last job, we were switching to Azure DevOps. They support Markdown in the developer workflow (PRs) but rich-text-only for Tasks (to be non-dev friendly?).
At my current job, I'm involved in developer productivity and have been interviewing people to collect input for where we want to take development. We currently use Phabricator which uses Remarkup. This is a source of frustration because its not quite the markdown they use everywhere else.
From these, I'm thinking Markdown Pages would be the top choice since it allows developer-only interactions and marketing-only interactions to stay within what they are comfortable with,
on (2) -> it sounded like you're saying (a) 'across entire suite identically is good', but that also (b) 'if there's a clear separation of Dev vs nonDev apps/docs, having different support is okay', did I read that right? please correct me if wrong.
In my personal opinion, Atlassian's rich text editor (or, seemingly, the 4-6 different one's you guys have built) is unforgivably terrible, so true markdown is the way to go if having both isn't an option.
The "reasonable subset of Markdown" is also a very useful specific detail, exactly the kind of specificity that helps us do our jobs.
It is so frustrating to have to remember to use double-curlies for inline code/pre sometimes, and backticks other times. In each case, the other doesn't work. More than once using one has resulted it rendering correctly on the immediate page but not on the next.
This is one I encounter regularly but is far from the only inconsistency in support.
Confluence's WYSIWYG editor will often make changes that can't be reversed with "undo" -- especially those involving indentation. Copy-paste frequently screws up its formatting as well.
So if you don't want to risk losing lots of work, you have to make many smaller changes. With each change taking a few seconds, it adds up quickly.
If it was markdown or some extended set of it, that wouldn't be a problem.
More generally, the issues I see reported only ever seem to be fixed in the Cloud version. I currently have to use the Data Center version.
Why go through the trouble of reporting an issue I'll never see fixed involving a feature that I loathe using?
> Why go through the trouble of reporting an issue I'll never see fixed involving a feature that I loathe using?
I find the same issue with Nessus Professional. They too are trying to funnel everyone into using tenable.io (SaaS nessus scanner). And they also are very resistant in doing much of any changes (other than removing the API) from their onprem solution. But tenable.io keeps getting regular updates.
Worse yet, when you talk to anybody there, the first thing is "Why arent you using tenable.io ?" My response every time is, "Has it been fedramped YET?".. Of course it hasn't. Not entirely sure they even plan on doing that.
For maintaining data integrity and preserving a secure environment, these companies are demanding that we open our networks and store our critical data somewhere we really don't have access to.
Editing an existing page is more perilous. I sometimes write out my changes in plaintext first, and then paste and format it into the doc piece by piece. Even that can get janky, though.
It's like Confluence is punishing my attempts to write documentation.
You've got to be kidding me. Have you used your own product? Going from my carefully tuned Server install to cloud Jira or Confluence is a night-and-day difference. The Cloud product is virtually unusable in comparison for any heavy Jira user.
You don't need "specifics", you need your performance engineering team to literally take over the entire company for 6 months to a year. No new features - nobody fucking needs them, the features 99% of your users use have been in the product for 5+ years already. Whatever you're PM'ing, cancel it, it's a waste of time in comparison to making the product not suck. The biggest source of losing users to some other product is going to be the sheer pain of continuing to use Atlassian....
Just make it usable. Halve the number of requests. Cache more things client-side. Do more server-side pre-processing so that a round-trip is not needed when I click on a menu.
I'm not looking forward to when I am forced to migrate my users to a more expensive and less performant experience. I and hundreds of thousands of other administrators will be experiencing months of user complaints because of the forced migration as it is; this is Atlassian's real chance to make it suck less in time.
That's something you could easily figure out yourself. E.g., just grabbing some random JIRA:
Opening an issue in that tracker takes 24 seconds for me. Twenty-four.
They polyfill text!
That's just absurd.
To add a data point, I get 26 seconds on gigabit fibre that is 3 ms network latency away from "hibernate.atlassian.net".
I started using JIRA in 2008. It was faster then.
If this is what the "cloud" version is going to be, then we will be looking for alternatives, even though we're an Aussie company and I would like to be supportive.
As for Confluence, the wiki is just ok. The editor is clumsy and occasionally I have to go into raw HTML just to get highlighting, bolding etc to work.
If Atlassian is going all-in on cloud, then it needs to realize that cloud isn't "run our software, but not on-prem". Just like MYOB had to learn, it needs to be rewritten so that the web front end is streamlined and cached separately to the underlying API.
On 100Mb fiber...
1. I click a link to an issue
2. I need to do something on that issue, so I attempt to click on a particular section to go make a modification
3. Bam, some background script has loaded, some new piece of content was shoved in, and what I clicked wasn't the thing I was expecting to click
Also, certain interactions within JIRA take far too many steps, and each one takes far too long to load, so it makes me dislike JIRA even more.
Project managers love JIRA, but engineers don't, because each time you make us wait we are less inclined to deal with the software that PM's need us to use so they know how things are going, so instead we get more meetings. If JIRA were fast, we could cut down on meetings.
Please make JIRA fast.
I think this is the core issue. It is simply not designed to be useful for developers, it’s designed for managing developers.
It’s the same issue with time reporting tools. The UI for entering data is just there because it needs to, but it’s not the central selling point of the software.
The UX for the data entry is just not designed to solve any problems besides accepting the data required for the reports that are the real product.
I spun up a free-tier account and created an empty issue. No data. No history. Nothing in any form fields. As blank as possible.
The only positive aspect is that most of the traffic is coming from a CDN that enables: Gzip, IPv6, HTTP/2, AES-GCM, and TLS 1.3. That's the basics taken care of.
Despite this, reloading the page with a warm cache took a whopping 5.5 seconds. There's an animated progress bar for the empty form!
This required 1.2 MB of uncacheable content to be transferred.
With the cache disabled (or cold), a total of 27.5 MB across 151 files taking 33 seconds is required to display the page. This takes over 5 MB of network traffic after compression. (Note that some corporate web proxies strip compression, so you can't rely on it working!)
For reference, it takes 1.6 seconds on the same computer to start Excel, and 8 seconds to load Visual Studio 2019 (including opening a project). That's four times faster than opening an issue ticket with a cold cache!
Meanwhile, the total text displayed on the screen is less than 1 KB, which means that the page has transfer-to-content efficiency ratio exceeding 1000-to-1. This isn't the animated menu of a computer game, it's a web form!
To render the page, a total of 4.35 seconds of CPU time was required on a gaming desktop PC to with a 3.80 GHz CPU. Having 6-cores doesn't seem to help performance, so don't assume upcoming higher-core CPUs will help in any way.
A developer on an ultraportable laptop running on battery over a WiFi link with a bad corporate proxy server in a different geo-region would likely get a much worse experience. Typically they might get as little as 1.5 GHz and 20 Mbps effective bandwidth, so I can see why people are complaining that Jira page loads are taking 10+ seconds!
In perfectly normal circumstances your customers are likely seeing load times approaching a solid minute.
PS: I do development, and I've avoided Atlassian products primarily because there's been a consistent theme to all discussions related to Atlassian, especially Jira: It's slow.
Stop asking your customers if they're running plugins, or what configuration they're using. Start asking yourself what you've done wrong, terribly, terribly wrong.
A lot of people explained that they do development using a Macbook Air on battery. Those are an order of magnitude slower than a plugged in desktop PC. Twenty milliseconds for me is a two hundred milliseconds for them!
Similarly, many outsourced developers are forced to work on cloud VMs that are not only relatively low-spec (1 or 2 cores), but outright throttled, such as the B-series Azure VMs.
Web developers at ISVs like Atlassian are also "spoiled" by having essentially unfettered LAN connectivity with 1-5 millisecond latencies. Worse still, they'll do development with the server component running on their localhost, which is basically cheating.
Real enterprise networks have at least two firewalls between end-users and the Internet, and at least one web proxy, which more than likely supports only TLS 1.2, HTTP 1.1, Gzip, etc...
We have customers that have less Internet uplink bandwidth for 15,000 users than I have for myself at home.
But all of this is immaterial to Jira's performance woes. It's slow in all circumstances. There is no way to make it fast. Not even liquid nitrogen cooled Zen 3 CPUs running at 8 GHz could bring the page load times down to what I would categorise as acceptable.
Indeed. This was a surprise for me too on Windows laptops. I think this was a change in the industry's approach to power management that happened some years ago, because I don't remember spotting these issues with the first two laptops that I've used. But in the past few years, input processing latency has became a telltale sign that I'm running on battery, on a "maximize battery life" profile.
1. Viewing a board. This can take 10+ seconds to load.
2. Dragging an issue from one column to another. This greys out the board, rendering it unreadable and unusable for 5-ish seconds.
3. Editing a field. I get a little spinner before it applies for 2-3s for even the simplest edits like adding a label.
4. Interacting with any board that crosses multiple projects. A single project board is bad enough, as in point 1, but we have a 5 project board that takes 20+ seconds.
Actually, I found an action that's pretty ok: Search results are fast, even if clicking into any of them is not. I'm not sure why rendering a board is so different performance wise.
Just to clarify on Search though, which search are you talking about:
a) quick search (top bar)
b) issue search (the one with the basic/JQL switcher)
c) something else
Trying to narrow down the latter "even if clicking into any of them is not" part to understand which view that is
I think I mean issue search.
By clicking into them, I mean actually loading the issues is slow.
Not just the Cloud service, the self-hosted versions are also painfully slow no matter what resources you throw at them.
That makes the other UX issues worse because the feedback loop has so much lag.
Data Center includes a few performance-related features like being able to run multiple frontends. I think we're running 4 instances right now. It's still really slow, even when nobody else is using it.
Also in Bitbucket. It used to be super fast, but recent changes made it very slow.
My team loves the integration with JIRA but we're considering going to Github because of the slowness.
Jira's git integration is really poor anyway.
Even redmine is better, where git commits that mention the issue get thier own column in the UI, and they ALL show up there, not just the ones with the comment annotation.
I would... but Atlassian TOS prohibit me from doing so :(
That's the problem. It's beyond repair since many, many years. You can only make it worse.
Ditch it! Throw it away, and rewrite from scratch. If you don't bungle it again (here lies the risk, as we're still talking about Atlassian) you'll have a better product then ever in one year.
And the first answer to your comment in this thread profiling performance for an empty page with almost no data on a small project should give you even more data than you ever would want.
And this one for an empty project: https://news.ycombinator.com/item?id=25616069
However, having personally experienced "upgrades to Jira and Confluence experiences" over the past few years, I can safely say: no one at Atlassian gives two craps about this. All the talk about "We are definitely working on generalized efforts to make 'most/all/everything' faster" is just that: talk. There's exactly zero priority given to performance issues in lieu of flashy visual upgrades which only make the actual experience worse.
> We're trying to focus on frustrating pages/experiences rather than number of network calls and such, because while the latter is correlated, the former (frustrating slow experiences) is really the bar and the higher priority.
Exactly: you aren't even trying to understand what people are telling you. These metrics you ask for and then dismiss entirely are the primary, core, systemic reason for frustrating slow experiences that you pretend are "high priority". No, frustrating slow experiences have not been a high priority for years (if ever).
If you need to do 200 requests and load 27.5 MB of data to display an empty page, therein lies your problem. You, and other PMs at Atlassian fail to understand these basic things, and yet we get platitudes like "performance is our top priority". It is not. You're good at hiding information and buttons behind multiple layers of clicks, each of which needs another 200 requests and 5-15 seconds to execute.
Oh. You're also good at adding useless crap like this: https://grumpy.website/post/0TcOcOFgL while making sure that your software is nigh unusable: https://twitter.com/dmitriid/status/888415958821416960 I imagine all performance tickets get dismissed because no one can see the description even on a 5k monitor
Can you guys please make Ctrl-S save and not exit editing? My muscle memory is costing me 10+ seconds of load times every time I type a paragraph and reflexively save, getting dumped back to the view mode of a document. The slow load times exacerbate the problem tremendously. I honestly don't know a single product that treats Ctrl/Cmd-S as "Save and Exit" so this is just a baffling UI/UX design decision.
This is interesting -> I don't think Ctrl+S should be exiting editing in any form, can you describe this a little bit further? When you hit this is it:
a) moving to Preview (you can see the changes you made, but in preview mode not page view mode (the page tree won't be visible)
b) browser returns to the 'view page' (page tree visible), but your changes are not published (you may see a tag at the top "UNPUBLISHED CHANGES")
c) something else (please describe)
Second question: what does the browser's forward/back buttons display after the 'ctrl+S' result?
The about page says "Confluence 6.15.2".
When I click "Edit" to edit a page, via `/pages/editpage.action?pageId=`, and hit Ctrl-S, it takes me back to the view mode rendered via `/display/SPACE/Page+Title`.
Absolutely infuriating and wastes an incredible amount of time. I can't get over my muscle memory. The "Save" button in the bottom-right does the same and appears to be what the hotkey activates. No mention of it being a "Save and Quit Editing". Hovering over the button says "Save your page (Ctrl-S)".
I will see if I can find some server folks to ask around, but obviously can't promise any movement.
If you’d like to see it for yourself, run Little Snitch in alert mode and try to sign into bitbucket.org - it’s almost comical how many hoops your browser jumps through.
*Confluence is built on Tomcat. Don't know if this is also true for Jira.
Now that my Confluence server is on Atlassian's cloud, it seems much slower still. So I have to assume it's not client-side JS because that hasn't changed much; there's some kind of resource starvation going on with Atlassian's servers.
I've used many others including Github, GitLab, Phabricator, Redmine, etc., and Clubhouse does a great job IMO.
We recently passed 10k stories and it still feels very snappy. Also the UI/UX feels very polished.
On a YouTube lecture at CMU the Materialize guys seemed to try very hard to not even land in the same zip code as that discussion to the point it was awkward, and they seem pretty smart.
Are these large organizations really that petty? Ad absurdism, would it be legal if Ford said we couldn’t drag race their car after we bought it?
You may not: disclose results of any Program benchmark tests without Oracle’s prior consent
It sounds like a bad business decision to not support open discussion on performance, it makes me think that they have something to hide.
>you will not and you have no right to: ... (f) perform or publish any benchmark tests or analyses relating to the Cloud Services without Cloudflare’s written consent;
Perhaps it should be more narrowly written, but prohibiting certain kinds of testing without permission is reasonable.
b) Rate limiting is practically mandatory in an era of multi-gigabit client network connections on even mobile phones.
Remember: It's not a DDoS attack if it's one user.
Edit: oh, and then they might dust off that Kali Linux VM lying around somewhere...
For Atlassian Cloud, you won‘t beed a benchmark to tell it‘s slow. A simple (physical) stopwatch is enough.
I'd personally find it pretty funny to go up to my boss and tell them I can't read my Jira tickets because Atlassian banned my account. Or would they ban the entire organization instead? Either way, hilarious.
I moved my team to clubhouse, even-though it created an element of fragmentation considering all other teams are using JIRA. I'm so happy though with clubhouse, I can actually get my work done without mindlessly waiting for every interaction.
Icing on the cake: I was considering moving the company over to Jira on a self hosted AWS instance, I've read that it can be a little faster. but.. they're discontinuing the self hosted option. Nail in the coffin for me.
Good bloody riddance.
Seeing that I'm not just the one who thinks it's so ridiculously frustratingly bad-slow [as in, it's impossible to be this slow even after multiple rewrites unless it's some kind of bizarre experiment by Douglas Adams' ghost] will likely lead to us dropping JIRA for good.
One of the most obvious examples: they have multiple WYSIWYG editor implementations which aren’t compatible. When you format something in Jira it’ll look fine in the preview and then render differently on reload. It’s been like that for years, nobody cares.
I spend 10 minutes to make a detailed bug report, just to have it fall apart after submitting. How does that happen in a software made to show bug reports?
Just use standard markdown instead of your own bullcrap formatting that doesn't ever seem to work.
I think the genesis of this is historically many different editable fields might had different modules behind them (not every single one a different one, but maybe a handful of common ones). It looks like for whatever reason we (Atlassian) haven't fully migrated all of them to the newest common editor modules -> I don't know anything for sure though
Atlassian is developed and run in Australia, right?
Have you tried running your website and doing the things we mention here from a VPN that goes through the United States or Europe to experience our latency to your servers overall?
I haven’t done any testing myself on this, but if you’re doing serial requests and each request is an https call back and forth to Australia, that’s easily 200ms every request, even if total server time is milliseconds.
And even if you support parallel, if you limit the number of requests per non-whitelisted IP by a lot, it can very easily become approximately serial
For the most part, each individual top line products' team operates out of a different site, with Confluence operating in Silicon Valley (mostly). I assume this is public information (or derivable) so I think it's safe to share.
Internally we have a few Confluence Cloud instances we use, with a single main one shared by the entire company (basically) which is located in a single area (somewhere?), though the VPN connection points are different.
So from a network topology standpoint our experience shouldn't be too different from most customers (at a very gross level) -> summary is, we definitely have users representative of 'bad networking', but you're right maybe I should be trying to intentionally degrade mine (right now i think its two cross US hops, but could be wrong).
I'll make sure our team looks at this dimension -> I don't think it's possible within our telemetry data, but maybe simulating it will get some interesting results.
I do recall reading once about something about AWS infrastructure (endpoints? edges?) having some configuration that causes omething like this:
1 - first returned packet is some some small size
2 - next packet (after ack is returned) can be double the size
3 - same
4 - same
5 - until max packet size
And though (1) is configurable (in theory), and I think the doubling/size increase is configurable (in theory), AWS does not allow for this configuration in their services.
But I can't find what I was reading so if anyone knows about that (a) being a problem and (b) how to work around it, let me know!
I self-host a confluence install, and it’s performance is poor even with a good sized VM and absolutely zero other traffic to it.
If you ban objective criticism, I guess all you're left with is FUD.
SPO is also responsive (the last thing to load is typically the SuiteNav which doesn't impact working with the site/content).
I'm not sure why a company like Atlassian would have these persistent performance issues.
I also got no response (or even an acknowledgement) for the feedback I gave. Like most people here, I too am forced to use it at work.
it is more like a general feeling all the time for many of us.
I'm using the latest Firefox on Windows, a developer laptop with 32GB memory that was brand new this spring as well as 500/500 fiber.
Right now on holiday (through Tuesday), but would like learn more. I see your contact info in your profile, if you give consent I'd like send you an email on Wednesday with some followup questions (the first question is "what is the URL for your cloud instance" so I don't want to be asking for it on a public forum)
On my company's cloud instance of Jira, it's a minimum of a 2-3 second delay to do anything. Edit, wait a few seconds, save, wait a few seconds, change a field, wait a few seconds... and God help you if you need to reload the page because something got stuck.
Sometimes things are just obviously crap to the people tasked with working on them and having to jump through these kind of hoops eventually leaves an organisation with only the kind of people who are happy to keep doing it.
Nothing makes me stop ignoring recruiter messages quite like being asked to flesh out a JIRA ticket with technical details by a person who has literally no use for the details they ask me to write.
Atlassian employees use their products, and tips and tricks they may have to use them effectively, or make the experience of using JIRA or Confluence, or their other tools more enjoyable and useable would be great to know!
If you have to use Jira or Confluence at work, you probably want to know how to make that as useful as possible. If you're working at Atlassian, you probably want to make your customer's experience as enjoyable as possible as soon as possible. Ideally you have a great product and great documentation and all happy customers. If that's not the case, you have an opportunity to work on a number of fronts, including improving documentation and the product, and help current customers with the product as it is. You can and should be doing all of these things.
Piling on doesn't help anything.
They don't want minor improvements to help it limp along, they want to vent and complain about it.
I think it's impossible that Atlassian evolves into a good product company, but it's entirely possible that my next CTO googled for opinions on the product, found a few discussions on HN with a combined 3,000 complaints about what a garbage fire it is, and went with Clubhouse.
Given your opinions regarding Atlassian, what would you think of your next CTO if they were even considering Atlassian? Is that someone you'd want to work for?
I really wouldn't give it a second thought, as everybody is using this stuff.
In my niche, there is very little focus on this kind of project management, due to the required speed of development and deployment.
If you work fast and reliably enough, delivering commercially important software, nobody is asking about a JIRA ticket.
My original point was that people should engage with each other in a way that's likely to create the most positive outcome. Creating throwaway accounts so one doesn't need to take the effort to be polite or take the time to be explicit about what they mean in a low-bandwidth context like HN is, in my opinion, highly unproductive and lowers the discourse on HN. If we're not trying to make things better, both the particulars of the situation (say, figuring out how to make Atlassian products easier to use) and put us in a better place for solving situations in the future (maintaining HN as a community people want to participate in), I don't think we should comment at all.
The throwaway I responded to hasn't participated further, and everyone else seems to have focused on the "tips and tricks" phrasing I employed, so one last attempt to elaborate:
1. You have to use Atlassian products at work for reasons that are out of your control. Your options:
- figure out how to make using the products as easy as possible.
- refuse to use the Atlassian products
- figure out how get your organization to change to another product.
- find a new job.
I'd argue only the first one is in your control as something you can do today on your own.
Some options for "figuring out how to make using the products as easy as possible":
- Complain on a forum that Atlassian products suck. The venting may make you feel better, but won't really improve the situation.
- Engage with an Atlassian employee
If you don't believe Atlassian is going to actually do anything, you might as well not make the effort of engaging at all. If you think there's a possibility you might find some relief by doing so, set yourself up for success. Snark and aimless, general complaints are unlikely to lead to a successful outcome, and I think it's likely to actively increase your likelihood of failure.
2. You're an Atlassian employee.
Note: If you don't believe Atlassian employees are operating in good faith, you can skip this section--and, for that matter, everything here. Get your company to switch tools, or quit.
Atlassian developers--just like any other developer--want clear, reproducible bug reports so you can fix the bugs (including slow performance). You want to know (a) what they wanted to accomplish; (b) exactly what they did; (c) what they expected to happen; and (d) exactly what did happen. If you want support, supply this information even if they don't ask for it 'cause that's what they need.
Fixing bugs takes time. Adding features takes time. Improving documentation takes time. All of those things should definitely be done. If the fixes are trivial, of course prefer the fix over the tips and tricks. If "tips and tricks" can work as a stop-gap to while these other things are worked on, by all means Atlassian employees should offer them and those using Atlassian products should use them if they want some relief now.
Time is a finite resource, and you need to figure out ways to move forward the best you know how. Your customers are likely diverse and have while they likely share some priorities, others are going to be different. Choosing to fix bugs A, B, and C while moving forward with features M, N, and O means that bugs D, E, F, G, H, and I and features P, Q, R, S, and T aren't going to be worked on, at least right now. And your customers that really want A, B, and C fixed and your customers who want features M, N, and O are going to be so grateful, and your customers who really a bitten by bugs D and F are going to be out in the cold, as are customers who want features P and Q. But if you can give customers affected by D a workaround in the meantime, I think that's better. That's just how things are, at any shop. And not just in software development.
If your priorities as a customer don't line up with those of the company whose product your using, your options are to wait, find a workaround, convince to the company to reprioritize, or find another product.
Focus on what you can do rather than what others should do. If you rely on the actions of others, it's still the same: what you can do to help others do what you want them to do--in a way that maximizes the likelihood of success.
The short version of all of this is engage with each other in good faith. If you don't believe the people you're working with are doing that and you don't think you can change it, it's really not worth continuing to engage with them, positively or negatively.
A key distinction is between domain complexity and what the product adds to that intrinsic complexity. If you use GitHub or GitLab issues with no customization you’ll have a better experience than a Jira user because they work well out of the box without requiring customization or adjusting your workflow just to accomplish core tasks.
On the other hand, I do not view those systems as products when it comes to professional use. A non-techie might see a Mac as a fancy laptop but for me it's a tool. Just like an HSS cutting tool in a lathe, you'd carefully maintain it (you don't want to use a dull cutting tool) and tune it. Just like you want to regrind a cutting tool depending on the part you are machining, I disable Intel Turbo Boost using http://tbswitcher.rugarciap.com when I run long perf evaluations of my programs for academic projects. If M1 based Macs will not allow me to disable their boost clock functionality, they will be unsuitable for my work as a tool. When that happens, you simply pick the most fitting tool. Not necessarily switching dev platforms, in this case a separate machine running Linux for the eval may be OK.
Regarding issue trackers, there are still some things I miss about Bugzilla after moving to Github issues such as sorting issues by two fields in order (eg first by prio and then milestone). I similarly like complex queries that can be saved in YouTrack. I will admit that 5 years ago I thought that Bugzilla was ugly (it still is) and not user-friendly (one of the worst) but now I simply see it as a professional tool that does not get in my way once I learn how to use it. On the other hand, most of the tools with proper UX (not all, most notably airplane cockpits have proper UX but still do not get in the way of a pilot doing their job including manual overrides for all kinds of malfunctions) have some "user journey" which gets in the way of almost every pro user.
We would love to fix "everything", and we have some longer term projects focused on this -> However, "everything" fixes seem to be a more incremental boost and also take longer time to complete.
If you have any feedback about "specific" items that are the most frustrating, we'd love to hear about those -> targeted fixes for specific can be much faster, much greater gains, and usually offer better user experience gain/engineering time returns.
If not, I can only say that we are definitely working on making 'everything faster'
(edit: trying to reply but looks like HN is limiting my reply rate)
(edit: maybe I can post my replies here and hopefully they'll get read)
@rusticpenn - This is definitely possible that 'some users are just used to it'. But we also see a very wide variance in individual customers' performance numbers (ie. some instances have consistently faster performance than other instances), and even within individual instances variance amongst users (some users have consistently faster experience than other users on the same instance) -> we're trying what we can to narrow down the causes in this variance.
Hearing from "users with slow experiences" is simply one of the ways we're trying to track this down, but it helps if users are willing to provide more info.
@ratww - thank you for the suggestion! We have some amount of data that helps us see what might be different between instances, but haven't gone out of our way to 'interview a fast customer', I'll bring this up with the team to see.
The two biggest factors I think we've seen: slow machines can contribute (but not a necessity), and large pages (especially with large tables, or large number of tables) can contribute.
What do your metrics show? I instrument my web sites so I know how long every operation – server responses, front-end JS changes, etc. – takes and can guide my development accordingly. You have a much larger budget and could be answering this question with hard data.
I’ll second the “everything” responses. Request Tracker on 1990s hardware was considerably faster than Jira is today - and better for serious use, too.
We have metrics, but of course as with many such things you always want more insights than the amount of data you're collecting (so we're always trying to grow this as appropriate).
This data is what led to the above (added edit trying to reply to @rusticpenn) saying we can see that "some instances are slower than others", and "some users are slower than others". I can't share those numbers of course though.
However, privacy reasons does prevent us from collecting too much data, so differentiating why individual users might have different experiences (even when other known factors are similar/identical) is difficult.
Also I'd be happy to take any suggestions you have about what to look at back to my engineering team, if you're willing to share other ideas. I know we're tracking several of the ones you mention but more options is always better.
Thank you for the advice - how about on the page load side? Our biggest problem is probably the variance issue (mentioned in other subthreads) -> we can't easily tell what is the difference between a slow and a fast load in many cases.
Even if we compare things that are available from the metrics like CPU, Mem, and Network speed, those are not very granular metrics (for example, to understand someone with 16 threads was actually at 95% Mem used during that page load), and have little correlation at a wide level with page load speed.
It is not your customers job to instrument your software. Your API gateway can provide precise and accurate figures on how long API calls take and there is nothing stopping you adding web page metrics that can provide client-side measurements as well.
Some examples are available on the publicly visible JIRA boards, like the one for hibernate. Just go click on all issues and then click on any issue in a private browser window and with the cache empty.
Every one of the fields take seconds to load. That is not internet roundtrip time, that is your backend. Even when the issue is ~80% loaded (according to your own page load bar), there are still JS scripts that will load and reformat the page, causing the browser to reflow.
These are not cached, because loading another issue doesn't resolve the problem.
So there are fundamental front end problems that have nothing to do with the servers or backend, they are entirely a problem of the JS and the in-browser activities.
I am sorry, there are probably customers who are used to the tools. Maybe they don't pass the Mom test. That point comes out as unnecessary defense here.
Can I suggest following up with those customers to see if and how they're using the product, what's their computer configuration, if there's anything special about them?
"This is very slow"
"We have no problems"
These aren't addressing the same things, really (unless the OP was translating "we're happy with the speed of the entire system" as "we have no problems").
Are the people reporting "no problems" actual end users. People I know who've become acclimated to Jira that I know would happily respond "we have no problems" while the people below them who have to use Jira 10x more often (multiple times per hour, vs a daily look at progress, for example) would happily say "this is slow as molasses (and that's a problem)".
Persistently asking for particular workflows, as you've been doing throughout this thread, shows a failure to understand the scope of the problem. In fact it makes me wonder if your paycheck depends on not understanding it. It sure seems like someone's does.
(i) publicly disseminate information regarding the performance of the Cloud Products;
But can we take a minute to talk about the combination of these two :
(h) use the Cloud Products for competitive analysis or to build competitive products;
(j) encourage or assist any third party to do any of the foregoing.
Does this mean as a jira user I can’t help build a jira competitor for a client of mine ( we are a tech agency). If this is the case I would really have a hard time using jira and being compliant. After all, who is the arbiter on what jira is exactly?But does this also mean people can’t do reviews on the platform as means to compare to other platforms? I’m a bit speechless here, it’s a wtf sort of thing.
Section 3.3: Benchmarking
Can you explain Atlassian's stance on Benchmarking?
Like many other software companies, Atlassian has this language in its terms to protect users from flawed reviews and benchmarks. By requiring reviewers to obtain Atlassian’s consent before publicly disseminating their results, Atlassian can confirm their methodology (i.e. latest release, type of web browser, etc.) and make sure they deliver verifiable performance results. Atlassian is not trying to prevent any users from internally assessing the performance of our products.
The language related to the public distribution of performance information has been included in our customer agreement since 2012.
Customers can obtain Atlassian's consent by filing a Support ticket. The Support engineer will then need to bring in a PMM for approval of the data/report.
This is implicitly recognized by allowing internal assessment: That assessment would be just as vulnerable to flawed methodology and therefore flawed decision making on products. If you were that concerned over such issues, you could issue further restrictions on performance assessment that limited such activity to be conducted only under Atlassian's close review or using your own mandated methodology. One reason you probably don't do that is because potential buyers would balk at those restrictions and either pass on your product or responsibly engage in due diligence and perform their own assessments regardless.
Further, the resources to do extensive internal assessment may be lacking in many organizations, which means your provision to allow internal testing is meaningless to many customers. As a result, the prohibition against public disclosure thereby deprives them of any way of obtaining objective external analysis.
You could satisfy your concerns by requiring that public disclosure be reviewed by Atlassian prior to publication. You could require an option for Atlassian to comment on the results with embedded notations without restricting publication itself. That would still be heavy handed but at least allow a reasonable amount of independent review of your performance.
DeWitt clauses are corporate censorship and are 100% self-serving.
There is zero benefit to consumers having no benchmarks at all available for entire product categories.
There is an enormous benefit to corporations to be able to silence critics with the threat of bankruptcy via lawsuit.
This is big corporations using the law to bully journalists and citizens, nothing more.
The solution to lies is not to censor, but transparency.
Atlassian has all the resources in the world to answer any external benchmarks done by third party.
If you can hire an army of lawyers, surely its possible to have a full-time engineer running benchmarks.
Perhaps California will address this problem by banning "performance benchmarking platforms" from listing evaluated products without an agreement from the vendor... 
When youve worked with azure devops or github before, atlassian tooling is really a blast from the past.
This is what happens when the "clueless" start "innovating". I've had several conversations over the years with members of Atlassian technical teams. They always wanted to work on performance, but never allowed (priorities).
They are in "good" company (Oracle, China, etc.).
What's preventing anonymous performance benchmarks, though?
For what it's worth, it's a pretty significant company priority now. I recently had my project (dark mode) cancelled so we could dedicate engineering effort to performance.