Most technical software in our industry requires weeks to get installed and running using your data. The vendor chalks this up to the slogan, “It was hard to build, it should be hard to use.” Yet there is a very real sense that when tools are complex, we assume they are powerful. Complex tools may be powerful, but not because of their complexity. When we sought to build Pinpoint we were forced into thinking through this tradeoff.

The first alpha version of Pinpoint was not friendly to use nor simple. Just ask the first few alpha testers. The journey thus began, how can we architect this tool to be simple, yet powerful. How can we make the tool so that teams can measure whatever they want to measure no matter how their flow is configured. We broke the tool apart into component pieces: a server and a client. The server to display the pages and a client to collect data. The client would run as a commandline tool so that it could be easily added into the flow. We developed a high level API for both in TCL, Python and Perl. We worked with a user-experience designer to design the web version to be both flexible and have good defaults.

The result after many iterations has allowed customers to get up and running with their data in the tool usually within a couple of hours. We’ve had people bring up the tool with very little interaction in India, Japan, France, Germany, and the US. Within a day they can have the tool integrated into their flow and start capturing data. If you’d like to see how easy it is, check out the How-To demos on our demo page.

We’ve made a lot of progress, but we’re not done. Our goal is to make things easier. Easier to communicate, easier to visualize, easier to track. This is a never-ending quest for us to continue to improve the overall experience.

One of our goals from the beginning of designing Pinpoint was to make the data useful for the engineers to use. Presenting the information both as a high level summary as well as allowing the engineer to dig into the details of what is happening on a various experiment. We’ve always provided engineers with the ability to label experimental runs, add comments to provide context, and hide snapshots that represent bogus data. In addition, we added the ability for engineers to mark certain snapshots as representing reviewed data and summarize this up to management as a status report.  All this to prevent people using the data for conclusions that may be inaccurate.

In Pinpoint 2.5, we added the ability for engineers to automatically mark snapshots as private. Even though we’ve seen that open communication among a team transforms communication, we wanted to make sure that the tool could be useful even when running wacky experiments that an engineer wants to keep private.This allows them to still check the data into Pinpoint and see the summary of the results or compare various experiments but not worry about any failed experiments being seen by others. Once the engineer decides that they would like to share the information, they can click the publish button and then share the snapshot with others.

Our goal continues to be helping to improve engineers lives and helping make teams as productive as possible. Helping to provide flexibility on managing the data is one of the ways we do this.

When we see data presented, we start looking for reasons that the data is that way. We want to know why. Why is the stock market up? We want to know and if no one helps us along that path, we’ll make up a reason. When teams start collecting data on their projects, the raw data is looked at by everyone, but only the people who are closest to the data know what the data means. We realized that when teams are working and management is looking at the current metrics of the project that there is a need to mark certain measurements as not only reviewed but also add comments.

This is one of the reasons we built the ability to add “best so far” flags into Pinpoint. This allows an engineer to flag a particular snapshot of the metrics as the best run so far as well as explain why things are the way they are. This flags a run so that management is only looking at runs that have been reviewed and are immediately provided with the associated narration that goes along with the data. Moreover, the running narration over the course of doing a design provides the story of an entire block through the design flow. By the end of a project, it’s difficult for everyone to remember what was going on 2 months ago, but being able to look back and remember can provide insights into how to do better on the next project.

How are you currently managing the narration that surrounds the metrics for your design?

Here they are:

1. Tapeout is like finals week at college except it lasts months instead of a week.
2. Tapeout is like having more to do than is possible to do, and you knowing will be judged by and large on how you deal with that.
3. Tapeout is like putting together a 7,000,000 piece jigsaw puzzle without a picture on the front of the box, while people come by invoking the power of ECO to rip out parts of the puzzle you just finished assembling, or load you up with piles of new pieces.
4. Tapeout is like reading a three hundred page book [of timing reports] that has no plot, every day, for a month.
5. Tapeout is like watching your “family” spend all your money on necessities and emergencies, knowing it’s your job to pay the mortgage at the end of the month, knowing the money won’t be there, knowing you’ll be on the street when it isn’t, and knowing everyone knows it’s your problem, not theirs.

This is important to us because we are trying to make it easier for teams to get to tapeout without the current number of headaches that keep them away from their family. Let us know how you describe tapeout and we’ll add it to the list.

I was giving a demo to the PD manager Vishy* and his boss Scott*. After showing some of the pieces, Scott said to Vishy,

“So this is the real data then. I should be able to look at this and tell whether or not we are almost done. This seems a little different from the status you were giving at our last meeting.”

Vishy responded, “Well, I’m not sure what data is checked in here. Some of this is probably old.”

Scott concluded, “Okay, so this tool is really only for your group’s use. I shouldn’t make any decisions based on what I’m seeing here.”

This is of course antithetical to Pinpoint’s purpose, but Scott’s point illuminates a critical problem:

In order for visualization to be valuable to management, the data must be clean and current.

Basic housecleaning can be pretty simple. Engineers need to prune failed experiments, correct specific metrics when a datum is flawed, and mark their “best so far” runs so that status reports show their actual progress, and not all the clutter of experiments. Adding a few notes to help put the numbers in context can also add a lot of value. Scrubbing data at this fundamental level gives management confidence that they are looking at real information they can use to make decisions. That leads us to the second critical problem.

The only way to ensure data is clean is to give engineers a good reason to clean it, regularly.

Most people don’t love to clean house for its own sake, so the scrubbing data has to be in the service of performing an engineer’s day-to-day job. This is where so many homegrown solutions fail. They automatically collect information but don’t provide tools that are useful for the engineering team. As a result, the data is a hodgepodge of random experiments, runs, and iterations — only slightly useful to the engineers and hardly ever useful to management.

The challenge we are eagerly tackling:

Make the data valuable by giving engineers compelling reasons to scrub it, all the time.

We all have a limit for the amount of things we can keep in our head at one time. We remember the mistakes that we made most recently, but tend to forget the ones that haven’t bitten us in a while. This can cause problems when you close chips, managing ever increasing complexity, hoping you’ll remember all the various gotcha’s.

The more complex the chip, the more each engineer must specialize, creating unique solutions to unique issues. Passing this learning on to the team is very difficult.

Complexity also brings unexpected side effects. Just this last week, we heard a story of a team who spent a good deal of time putting in place clock gating for the whole chip. Things looked great until they pulled in the logic from the BIST module which introduced an extra 20,000 ungated flops into the design. It was a power nightmare.

Key problems facing teams go beyond the difficulty of tracking metrics. They must also make sure the entire team can see the effect of each member’s changes on the chip at large as well as provide a mechanism to capture organizational learning so that past mistakes are not repeated.

Earlier in our life as a company, our practice was to assign each engineer the responsibility to deliver what were essentially independent features.  This worked okay, but it had several big drawbacks. First, it isolated the engineers into their own thinking. Sometimes there would be some coordination between a couple of engineers where features overlapped, but mostly each engineer worked independently and developed isolated solutions that only he or she understood. Helpful feedback between peers could be a big challenge.  Second, the engineer who took the longest, delayed the overall release.

We got the job done, but over the last six months we’ve changed our approach, and it’s increasing our productivity and strength as a team. By breaking each major feature down into it’s component pieces, and then assigning the various engineers to work on parts of those pieces, the team is working better than ever. Besides minimizing the two costs above, we develop better code faster.  Since people can pick up slack for each other, and since the team shares a more global understanding of each feature because essentially we all work on many parts of many features, the excitement of everyone involved increases.

Visiting our Fort Collins Engineering office, you can sense the enthusiasm of the team.  They work fluidly, collaboratively, and together get a whole lot done. We also see it happen with our customers, Physical Design teams who have the tools they need to better collaborate and share their work. And when a team is working together toward a common goal, it’s 1+1 = 10 (not just in binary thing, either).