December 2020 Competitiveness Improvement Project Workshop (Text Version)

[Transcription begins at 0:00:40]

Moderator: Good morning, everybody. We'll be starting up in about a minute. Give folks 5 minutes to get here; then we'll dive into the presentations.

So, welcome to the—to those on the phone, welcome to the 2020 Competitiveness Improvement Project Workshop. So, we'll get going in just a second.

[Silence from 0:01:10 to 0:02:40]

All right, everybody. Why don't we get started? Welcome, everybody, to the Competitiveness Improvement Project 2020 Informational Workshop. Apologies, everybody. I mean, these events have always been great in the past because we've had a chance to kind of meet in a room and talk about CIP as well as talk to each other, and so we're saddened by the fact that we can't invite everybody up to the National Wind Technology Center at the Flatirons Campus and kind of talk through these projects. But this, I guess, is the next best thing. And hopefully we're trying to make the best of a new situation and we'll get people in attendance who would not have been able to be in attendance to be _____.

We have had to kind of readjust the agenda to make it fit more into a virtual space, so it's a little bit shorter than previously. But one of the added benefits is we've got a section that I'll get to as I walk through the agenda in a little bit. We're going to do some quick introductions. Patrick Gilman has been delayed slightly, and so he'll be here in a minute, and when he arrives he'd like to welcome everybody. But Patrick Gilman from the Department of Energy, who oversees with Bret Barker the distributed wind research portfolio for the Wind Energy Technology Office. So, he'll give a few remarks. And then I'm Ian Baring-Gould from NREL and I oversee the CIP of distributed wind research portfolio from an NREL perspective.

Because we don't really have time to do around-the-room introductions—that would just take too long—what I would encourage everybody to do is to type into the chat a little bit about your company and yourself so that we get a better sense of who is on the phone and what you're doing. So I suggested your company, company website, and contact information and then a little short sentence about what you're doing in CIP, kind of your engagement with CIP. Again, one of the great benefits of the in-person meeting is the ability for you to meet people and kind of have conversations, sidebar conversations in the hallway. And so, that's a little bit difficult in the virtual settings, but we want people to be able to engage in dialogues to the extent possible.

Quickly looking at the agenda, we've tried to kind of fit this in into somewhat regular business hours from coast to coast. So, we'll have these welcome and introductions, a quick overview of CIP, and then we dive into a bunch of different presentations of topics that we feel will be useful in—for companies that are developing or thinking about developing potential CIP proposals going into the future. So, a fair kind of [garbled audio]. Brent Summerville, who I'm very happy to kind of let folks know that is now an NREL employee, he just started 2 weeks ago, but he'll be talking about the new SWT-1 standard, which changes the dialogue a little bit in regards to the certification testing and things like that as we go forward. Dean Davis from Windward Engineering will give us kind of a similar presentation to what he gave us last year, but really talking about certification testing requirements and some of the details and complications around that certification testing.

We do have breakout areas on potential topic areas—so, breakouts on potential topic areas, which is new from last year. And this will allow folks to kind of identify topical areas they might be interested in, and then we'll have people go into breakout sessions where you can talk to NREL folks but then also other companies that are interested in those specific topical areas. So, that should allow us a little bit more detailed conversation on each of the topical areas, which hopefully will be helpful.

After kind of a short lunch break—we will lunch depending on where you are—we'll have some kind of discussion and Q&A. We really value all of your input into the CIP project and the CIP process and so would encourage anybody and everybody who has comments or thoughts in regards to ways to improve or change CIP to provide them. We're always open to learning how we can do this better to support all of you, the distributed wind industry.

Kyndall will then talk about procurement and contracting, some of the highlights that we need to keep track of there. Robert will talk about the design evaluation. And then we've got kind of a potpourri of topics at the end, kind of additional considerations, and that will be Robert and Scott Dana talking about those, and then we'll have some Q&A at the end.

Just some things: Please mute your phones and keep them muted. You should have come in when you were muted. There is a Q&A feature, and so please use that. The little tab is up in the upper-right corner. Please use the Q&A feature if you have questions. Certainly use the chat function if you have things, just comments or things you'd like to say. that would be great. But use the Q&A and we'll try depending on timing and things like that to not read your questions, but if you type in your question, then we know what it is and people can think about them, and then during the Q&A session—sessions—we can call on you to ask your questions. And so, we want to keep that kind of two-way dialogue going, but the Q&A feature is a good way to capture those.

As I said, we will do the breakouts kind of right before the lunch break. We're planning on four of them, so the kind of focus on the different types of proposals that you might be thinking of, the pre-prototype development component; innovation and system optimization; prototype testing and kind of the whole suite of testing efforts; and then manufacturing process innovation as the last one.

And so, if you go—and I'll go over this again before the breakout session, but if you go up to the top of your screen in the kind of file listings there is a topic called "breakout," and you can click on that and you can select which breakout you want to go into. And so, once we get to that point you'll go up there and just select which breakout you want to be part of. And then, obviously, if you don't want to be part of a breakout then you just get a longer lunch break. But think about which ones you would like to kind of be interested in looking at and taking part in so that when we get to that you can just hop over to it.

As I said, we really regret the inability to allow kind of expanded cross-industry dialogue, which is one of the wonderful things about an in-person workshop. You can under that breakout box create your own breakout. And so, if you would like to pull aside someone, hopefully not breaking into too much of the dialogue, you do have the ability to kind of pull them in or request that they pop into a breakout session, have a little sidebar dialogue during breaks or what have you. Now, obviously you can always pick up the phone and call them, but we're trying to keep this as kind of interactive as possible. So, don't hesitate to do that.

And then you call also chat or e-mail Rachel, who is our host today, our WebEx master, and she'll be able to work through any of the kind of complications that you might have.

Just a couple of quick notices, and then we'll kind of move forward into the introductions. NREL did announce a notice of intent on the ninth stating that we intend to release an RFP for CIP in 2021, early in 2021. So, there is a notice of intent on the street. That doesn't mean it will happen, and it doesn't mean it will happen early in 2021, but to a degree you've been warned. That being said, or with that kind of out there, NREL and DOE employees can't discuss any potential CIP solicitations, but we can provide guidance based on previous releases. And so, if someone asks me, "Hey, Ian, when is the release coming out?" the answer is "I don't know when the release is coming out. Or "Hey, Ian, is topical area two going to be included this year?" I can't tell you that. But what we can talk about at any length is kind of information related to all of the other CIP releases that have happened in the past and that kind of old adage that "The past is a good indication of the future." So, hopefully you'll find this useful. But I'll just let you know that we can't do stuff looking into the future and what potential future offerings might have.

We are recording this session and we will make the slides available to folks. They will be published. So, you'll be able to come back and look at these and you'll be able to view the slides once they're published for everybody to take a look at. And two other things: If you are not on the CIP list as of yet, please e-mail Kyndall and she can—she'll talk later today, but please e-mail Kyndall and let her know that you're interested in receiving CIP solicitations and the like. We do send out e-mails and things of that nature to make sure that you're kind of up to speed on what's happening. And then, always the NREL CIP website has a bunch of links, such as the one to the notice of intent. So, if you haven't seen that you can go to the CIP—NREL CIP website and get links to things that are happening going into the future.

I don't think Patrick has joined us yet. Patrick, are you on? I don't see him. OK. In which case I'll kind of keep going, and then when Patrick is able to join us we'll kind of break away and he can do a quick introduction and welcome.

Male: Ian, this is Bret. I can do the introduction for Patrick. He's having technical difficulties getting in. He let me know by text.

Moderator: OK. So, Bret, why don't you give a quick introduction? I introduced you a little bit, but a little broader introduction. And then, yeah, if you could do the welcome that would be fabulous. Thank you. Thank you for that.

Male: Yeah, sure. Sure. So, I'll try and keep this brief, knowing that we're a little bit behind schedule here. But—so, yeah, Bret Barker. I help manage the Distributed Wind Resource Program at DOE which funds NREL to run the CIP. I believe we're coming into our eighth cycle of the CIP, which I think is really a testament to the program [inaudible due to background noise] we don't hang around that long unless they're really impactful. So, I think it's a testament to NREL and the quality job they do in running the program, but also a testament to the distributed wind industry, who continues to develop really impactful proposals. So, I want to thank both NREL for putting this together and also the industry participants for their interest and participation today.

I think really this is an opportunity for newcomers to be introduced to the CIP or repeat applicants to get reintroduced to the process and learn from our experts at NREL about how to be successful within the program. But it's also an opportunity for you to provide us feedback on it. And as Ian said, we can't discuss anything related to the NOI and the anticipated announcement, but if we look at history and prior topic areas, you can certainly provide feedback based on those.

So, with the—we look forward to a successful workshop today. And with that I'll kick it back to you, Ian.

Moderator: Great. Thank you, Bret. We certainly work very closely with Patrick and so encourage any and all of you to reach out to them if there are things that you want to address with DOE that it doesn't make sense to address with us. But they're a good team to be part of.

Just echoing, before I jump into the first session, echoing Bret's comments that our goal through this session, as well as kind of any dialogue we have, is to help you make good proposals. So, all of us win and you win if you submit a good proposal. The last thing we want you to do is spend a lot of time developing a proposal and have it end up being in the wrong topical area or not being getting well or something like that, because that's in truth a waste of your time and we don't want that. So, the goal of this as well as any other dialogues that happen is to help you kind of move your industry or your company along in the best way possible. And so, our goal here is to make sure we have the best proposals, we help you make the best proposals we can. And so, please don't hesitate to kind of put questions in or comments or anything of that nature. We'll try and get to all of them.

So, let me quickly—for those of you that are kind of new to CIP or at least relatively new to CIP, I want to quickly run through this kind of introduction. We'll go through this quickly and then we'll get into the meat of the presentations as we go through the day.

So, basically, the challenge that we're facing here is we need to lower the cost of distributed wind technology to be able to compete with other distributed technologies—photovoltaics as an example—but as well as build consumer confidence of not-as-successful past deployments. And we need to do this through rapid innovation. And so, CIP is really designed around those kind of tenets. How do we help the industry move forward quickly and innovate their technology to lower the cost, increase the reliability, increase the power production so that we get a lower cost of energy that allows distributed wind to be more kind of equally or more fairly against other distributed energy technologies—again, like photovoltaics?

And so, that's a combination of innovation around technology, certification, but then also supporting a very broad development pathway in regards to the development of distributed wind technology. Generally speaking, we want to expand the US leadership in domestic and international distributed wind across the board. And so, CIP is open across a pretty large range of technologies, or has been in the past, and we want to keep it that way. So, this is not something that is specifically identified to small wind, nor are there any topologies or applications or anything of that nature that are more favored or less favored than one another.

CIP, as Bret mentioned, we've been doing it for quite some time, engaging with a lot of companies across the United States, and have invested a good amount of _____ leverages, a good amount of your money, which is fabulous. The last little bit is that NREL manages the projects. It's a DOE program that NREL manages. And then also the laboratories—primary NREL but all the laboratories—can provide technical support to companies that receive CIP contracts to move forward. These are competitively solicited contracts, so they're not grants, so there is a reporting process that goes along with it. But it does allow us to provide more active support for the projects as companies need it.

Generally, the approach to CIP is—kind of builds off these following concepts. As I mentioned, they're competitively selected and cost shared. And so, DOE puts in the lion's share of the money for these but each of the companies need to put money in, get skin in the game to move these forward. Generally, we focus on relatively short contracts, and the idea here is to support companies as they move through the development process—and I'll get to that in just a second. And we really want the contracts to be completed in a timely manner. Historically, we've been able to do a CIP release every year, and so that allows companies to kind of develop and progress as they go forward, learn, evolve, and then always, or generally always have the ability to apply for new funding every year to move their development process forward. And we've been very successful in—or, you've been very successful in kind of taking that approach.

We really are focused on higher technology readiness level efforts. And so, the idea here is to get new products onto the street or innovate existing products to lower the cost of energy through lowering cost or increasing performance. And that's a really core focus. CIP is not an R&D research project. It is not for people who have some wild idea that they would like some money to understand whether it makes sense. This is for companies that have really solid ideas that they are moving forward with and want help to be able to do that.

We are neutral in regards to technology, configuration, component, application. So, if you're—if you've done an analysis of your system and the foundation is your highest cost element, well then, submit a proposal looking at the foundation. It doesn't have to be for a wind turbine itself; it just needs to be related to wind turbines. If you want to be looking at different applications for wind turbines—so, off-grid applications or things, telecom, something that's a little bit more unusual—again, it doesn't have to relate to the wind turbine itself. We're interested in moving the distributed wind industry forward, and the applications for which your turbines are applied is completely applicable to that.

And then also, lastly, configuration. Vertical axis, horizontal axis, kind of innovative wind power designs, all of those are applicable within the CIP context. We're looking for good ideas that have a path to certification and a path to the market.

If the—I mentioned this, but the idea is to really get to turbines and market. So, innovative applications but certified turbines. So, that's where we want to get. If we get to a point where we can't move forwards, whether that's kind of a pre-prototype and we kind of get—you and us get to a point where we determine you can't get a marketable turbine, or if you start a certification process and we go through the design review and you're not ready for certification, we look to end the contract. And we'll do it in kind of a collaborative fashion. Again, it gets to this question of we don't want to waste your money and you don't want to waste taxpayer money. And so, if the kind of general scope of the contract isn't achievable, then we'll end the contract and that's not a bad thing. That's actually a good thing. And that wouldn't keep you from applying for a future round of CIP. Our goal here is to move the industry forward, and that means we need to be flexible.

And then, as I mentioned, we do provide technical support. Generally speaking, your proposals need to be complete, and so it's—you can't just kind of hand off some of the responsibilities to NREL or one of the other laboratories. And generally speaking, we want to use the wider wind industry to do any work that is being discussed. If there are things—and we'll talk a little bit about [audio garbled] specific to the laboratories, core capabilities that the laboratories have, either kind of a staffing person or equipment [audio garbled]. But if you can get services within the industry, then we want to support the industry and not support work at the laboratories, which is another kind of core tenet of CIP. So, as we go forwards there is a place within the proposals to identify work that you might want the laboratories to do, but just make sure that that work is really something that only the laboratories can do. And if you can find someone else to do it, then that's all better for the industry.

So, historically we've had these CIP topics. And again, the breakout sessions will dive into each one of these in a little bit more detail. The pre-prototype development is mainly focused at turbines that have not had a prototype built. And so, you have a really good idea and you want to build that idea out, you've put it in a wind tunnel, you've run it on the roof of your car, you have a good idea and you have documentation around the fact that it is a good idea, but you need some resources to take that to the next step. And that's what the pre-prototype is really designed around. The goal of the end of the pre-prototype is to have a paper document that is a detailed design specification of your turbine so that you can move into the prototype testing. And so, that's really for early stage technology but it's not for wild technologies. We get a bunch of applications—for the last couple of years when we've done this we've gotten a lot of applications, and so it's really competitive and you need to kind of really—kind of demonstrate and document that your idea is very solid. But if you can, then this is an opportunity for you.

Prototype testing is as it says. You have a prototype and you can put it up on top of a tower, but there are some issues that you're having with it that you want to work through. Maybe you don't have the power electronics package really built out. Or you've run it for a while and the generator is just not putting out as much as the model said it should. Or you need to really work on that—those—that tip mechanism or the rotor needs to be refined, your pitch mechanism isn't working. Something of that nature. And so, you have a turbine that can go up on a tower but you have some real things that you want to work through in regards to that prototype and you know you're not ready for certification.

Overall system optimization is looking at the total system or a part of your turbine that includes a number of different parts. So, it could be the drivetrain. It could be the power—kind of controls, power electronics inverters. It could be the whole rotor assembly with pitch mechanisms. And so, this is a little different from component innovation, where are taking a specific _____—the tower, the blade, the foundation, and you want to innovate and lower the cost of those. The system optimization is really designed around a number of interlocking systems that you need to look at at a [audio dropout].

Now, you could certainly go through this concept of doing prototype testing and then want to go in and do an overall system optimization. You've done the prototype testing, you've figured out what you need to do, but you realized that your turbine is twice as much—kind of twice as costly as photovoltaics and you really need to optimize the whole system to lower the cost. That would be a fine application for component optimization—or, sorry, overall system optimization. Or you've got a turbine that you've been operating for ten years that's a solid turbine but you need to take a look at the whole system to be able to lower the cost. And you can do that through system optimization.

I talked a little bit about innovation, component innovation. That's when you have a specific component—tower, blades, what have you. And then we have two types of testing, or we've had two types of testing in the past. One for turbines that are smaller than 200 meters squared, and then one for turbines over 200 meters squared. Brent Summerville is going to talk a little bit about the new standard, SWT-1, that kind of—that obviously is something that is going to need to be addressed in future solicitations, so I'd encourage you to kind of plug into that presentation. But we have broken those out separately because the US and international standards do separate those two classifications.

And then last but not least, manufacturing process innovation—or, one last thing, I guess. Both those certification testing and type certification, as we'll hear later today, are pretty expensive processes, and so part of kind of putting in applications for those is making sure that your turbine is ready, because once you've invested in that and we've invested in that the last thing you want to do is make a design change that requires you to go through recertification. Nor do you want to start a certification process unless you're pretty sure, 99 percent sure that you will finish it. And so, that is a clear element of both the certification testing and the type certification, is making sure you understand what it means to certify the turbine. And then we do have some checks throughout the process to be able to kind of go through those design evaluations. And then if we all agree that the turbine isn't ready for certification, then you have an opportunity to pop out before we all invest the money to do something that in the end is going to look like a bad idea.

Not all topics are—have been offered in the past, and we certainly include feedback from all of you in regards to new and different topics that we could add to this. We're always looking for feedback, so we would encourage that.

Last but not least, until an RFP is on the street we're more than happy to talk to you. Once the RFP is on the street then legally we're not allowed to. So, if you have questions about which category you might fit into, then definitely call, reach out to any one of us, and we'll be able to have a conversation and talk through that with you. Once the RFP is released—which, again, we identified in the notice of intent was going to be early in 2021—we are not able to engage with you. So, reach out early and start those dialogues.

Kind of examples that are within scope based on past. Pre-prototype turbine assessments. So, again, you've got a good idea of a turbine, and to a degree we've had a lot of kind of interesting ideas. And so, vertical, horizontal. We haven't had any airborne wind or anything of that nature, but all of those types of turbine concepts are certainly encouraged. And the real idea here is to help you kind of finalize those initial designs and get them to a prototype development stage.

How to optimize components or overall systems. So, the overall system optimization, how to kind of look at a whole combination, redesign the whole power electronics, put storage onto the DC link of your turbine—which is certainly going to impact both kind of the drivetrain but also the controls and the power electronics of your turbine. So, things of that nature are certainly available.

Improvement of components. And again, these don't have to be on the turbine itself. It can be inverters. We've done a lot of inverter work, as we'll hear about. Blades, towers, foundations. Even components that get to this application—so, Bergey Windpower did a deployable turbine—or is working on a deployable turbine, which allows it to fold up and so that it can be moved on a trailer. So, we're certainly looking at applications as well for turbines as well as components of the turbines themselves.

We will support all types and sizes—and have supported all types and sizes—of different turbines. So, micro, small, medium. As I mentioned, horizontal, vertical. Unconventional designs or configurations. All of that is open and we encourage applications. Again, the key is to demonstrate to a bunch of geeky engineers that you have done your homework and you have a solid process to a product. That's what's the key.

I already talked about foundations. I talked about the different types of testing that we allow: the pre-prototype with a good plan, and then moving towards certification. And then, we have funded manufacturing projects, a number of manufacturing projects as well.

So, stuff that we—that are kind of out of scope and—I kind of touched on this—lacking technical merit. If you can't provide technical merits through a proposal then the geeky engineers that are going to review this are going to say, "You have no technical merit." And that's the fastest way to fail. So, certainly don't claim you get above the Betz limit and thing of that nature. And make sure that you've done your homework. You're wasting your time in writing a proposal if you cannot justify the product as we go forwards.

Not addressing specific topics identified within the proposal—kind of incomplete proposals—really gets to the credibility of the people that are writing the proposal as well as your technology. Certainly, if all you have to put your hat on is a website, video, a marketing brochure, you're not ready for CIP. The people who review the proposals have upwards of 50 years of distributed wind expertise, and so you're going to have to make that justification.

You have to be partnered with a turbine, and that is because we don't want people proposing kind of nifty ideas that don't have a path to market. And so, if you have a new blade design or something of that nature you need to have a small wind manufacturer that is—that states publicly and within the proposal that they will use your technology on their turbine, because we don't want to put a lot of money into a bunch of different neat ideas that have no path to market. Right now, at least historically speaking, we haven't funded work for very large turbines, and that will continue going into the future as far as we can tell.

You certainly need to have a U.S. presence. You have to have—you have to be a U.S. corporate entity, and that's by contracting rules, but that doesn't mean the turbine has to be made in the United States or something of that nature. You do have to demonstrate active participation and impact on the U.S. market. This is U.S. taxpayer money, and so we have to be able to walk that back to the impact on the U.S. And to the extent—well, not really to the extent possible, but a major amount of the work should be done in the U.S. Now, all of the contracts do require cost share, and so if your cost share is supporting efforts that happen internationally, that's your corporate business. But to the degree that we can, the money that is the US taxpayer money should be spent in the United States.

Clearly, we're not looking for unsolicited proposals that go outside of the scope of the topical areas. So, it has to fit within a topical area to be able to be considered. And then, we also don't—won't support proposals that are for multiple topic areas, and so that would be someone who says, "Well, I'm really close to certification but I want to do six months of prototype testing and finalize this power converter and then I will start certification testing." That's not acceptable. If you're in a prototype testing stage then submit for prototype testing. If you're ready for certification submit for certification. Don't mix and match different topical areas. And that's one reason that we do this every year, or to the extent that we can. We do this every year so that if you're ready for prototype testing, well then, submit for prototype testing. And if we're able to,—but all of our plans and our expectations allow us to do this, Congress-willing—we'll have another CIP in a year, in which case you can apply for the certification testing at that point.

Now, certainly you can apply to multiple topic areas for the same turbine platform. You just have to be really careful that they're not competing. So, if you want to look at a tower and blades as an example, as two innovations, and those two concepts aren't competing, then go ahead. Please submit two proposals. Better than trying to submit one proposal for two—for the blades and the towers, which again, as separate innovations, that will create some problems. But if you want to put in two proposals for innovations on your blades and on your tower, just so long as you can do either one separately assuming that both aren't awarded, then that's perfectly reasonable.

We really do not want to—we do have some flexibility as we go into the contracting process, but we really do not want to modify proposals very much. And this gets back to the fairness amongst all of you. All of you, as you will see or have seen in the RFP documents from last year, the scoring criteria is dependent on which topical area you're addressing. And so, if you submit something into certification testing and start that, as an example, and then say, "Well, in truth I'm not ready for certification testing; can you switch this to a prototype testing so that we can work out the details?" that's a different scoring criteria. And someone who put in a proposal for prototype testing would probably feel a little bit unfaired against if you—if one—Company A submitted a proposal for something different and then ended up with a prototype testing.

So, we can shift stuff a little bit within the topics based on justification. So, the approach to addressing innovation around a component, switching between vendors, potentially changing the place where you're going to do your testing if there's a reason to do that, adjusting project plans, staffing plans—all of that kind of stuff is available. But if you want to submit for one area and in the end you want to go from a component improvement to a system optimization, we are not going to be supportive of that. And so, that requires you to really understand what you're submitting for. In a case like that what we'll do is we'll push to end the contract and then encourage you to submit a new proposal going forwards to focus on what you really want to do.

Certainly, changing the primary vendor—so, if you're a blade manufacturer and you submit a proposal with Company X and then after six months Company X decides they don't want to play with you anymore, you can't say, "Well, OK, I'm not going to design my blade—or my turbine for Company X; I'm going to do it for Company Y." That's not going to be acceptable. OK? So, make sure you have your partners, your kind of primary partners well defined. And then clearly, if there are huge cost changes that's going to be a problem as we go into the contracting, after we go into the contracting phase.

I just wanted to give some quick highlights, just two examples of the criteria that we use. Looking to the last RFP, each of the topical areas have a very clearly defined criteria and a very defined list of questions that we are asking. Answer the questions. If you don't answer one of these questions you will get zero score. And if you get zero score I can guarantee you you will not be successful. CIP is very competitive, and if you don't address a criteria you don't get scored for that criteria, and if you don't get scored then you fail. OK?

This kind of top bullet is really important. Even if you don't have a strong answer to a criteria, well, then, put your weak answer in and that is acceptable. Better to put something in, acknowledge a weakness in your design, say that "We aren't quite sure what's happening with our inverter but here are some of the ideas that we want to pursue, and that's why we want to do prototype testing." That's perfectly reasonable. And it's better to put something in there than just leave a criteria blank. And so, think about that as you develop your proposals. But go back in and make sure you really understand the criteria, the scoring criteria, and write your proposal given that scoring criteria.

Some kind of final—getting to the end here—some final considerations and recommendations. If you're making incredible claims you'd better have really strong evidence to back up those claims. Proposals that kind of launch into a tirade about large wind or something of that nature, don't bother. This is all very professional and we're looking for really good idea. So, if you have good ideas we want to hear from you. If you don't have good ideas then don't waste your time writing a proposal. OK?

Technical expertise should be demonstrated in the topical areas that your proposal is looking at. So, if you're doing inverter work and you don't have a good power electronics person, well, that's going to be a big problem. Now, if you're planning on hiring consultants—because a lot of your companies are small—and you don't know exactly who that consultant is, I think that's fine. Or that is fine. Just make sure you really identify the skill sets that you're looking on and potentially comment that you have engaged with a number of people but until you are selected you're certainly not going to bring those people in on retainer. So, just identify who you would bring in to be able to complete the contract, and that's—the contract work, and I think—and that is fine. Much better than—that than leaving kind of classifications of people off.

Again, restating you don't need to be a wind turbine manufacturer to apply for CIP but you have to have a wind turbine manufacturer as part of your team. Almost all of the CIP efforts have go/no-go decisions that are kind of built into the proposals, and you will see that looking back at last year's. There are a bunch of kind of decision points and you should kind of plan your proposals around those decision points. This is especially true in the pre-prototype development, which only guarantees funding for the first part, and then you get to the design review and then we go forwards after that. So, think about how you would write the proposal, and then as you're looking into your corporate decisions keep that in mind.

You have to have a path to certification in anything that you do. Even if you're doing a component improvement on a design make sure that you have talked to a certification agent or people who are very familiar with certification to understand and document that path to certification. If you can't get certified then it's not a viable product. And so, even if you're in the very early phases of design, going back to the pre-prototype development, you should have certification in mind as you start doing that early product development, and you should be designing your turbine to meet certification, not designing your turbine and then when you get to the last inning of the game and you have to certify the turbine figure out how you're going to certify it then. You have to start that process early and we encourage you to do that. And under CIP you can get paid to do it, so don't hesitate.

Let's see. This last point, I kind of addressed it already, but there are kind of—look through the proposals. Make sure you understand the proposals—or the requirements or requests for proposals—and articulate what you are going to do. If there is something that doesn't quite fit in, just so long as you articulate what that is, then we'll review it critically. If you try and do something but don't articulate why you're going to do it or the reasons behind it or why you think it makes sense, then you leave the reviewers guessing. And in a proposal review process leaving the reviewers guessing is never a good thing. So, all of the proposals have kind of specific requirements. If you want to go outside of that it's OK. Just articulate it, and then we'll grade it based on what we see.

So, I think—I don't have the agenda in front of me—we're a little bit late. Yeah, five minutes late. So, I will look for questions, but why don't we move on to session two. We do have several series of Q&As later today, so if you have specific questions on what I've covered jot them down or put them in the Q&A and we'll come back to them.

So, let's move on to session two with Rob and Robert. Robert and Robert.

Male: Good morning. Can we get session two up? Let's see, can I do that? Yes.

Moderator: So, I haven't—I think you can get control or I can just type through your slides.

Male: OK. Let's see. I've got my—I've got it up on my screen. Is it not showing up on yours?

Moderator: We've got session two up and you have now been given control of the slides, so you can —

Male: OK, great.

Moderator: Within the thing you can move forwards. If it doesn't work just let me know and…

Male: OK. And Rob Wills, are you online here? Can you—if you are, can you unmute yourself?

It looks with your icon, Rob, like you're speaking but we can't hear you.

OK.

OK, well, I'm going to carry on. Let's see. Oh. There we go.

There are a lot of standards that apply to distributed wind, and so this is not going to be a deep dive into any of them but it's more about giving you an understanding of why there are so many different standards and different areas and how they build together for a structure. As far as turbine certification, which is basically about the performance and the structural integrity, that comes under either an AWEA standard, SWT-1, which is further going to be discussed later in other sessions, and the International IEC 61400-2 or -1. -2 is for turbines with a rotor-swept area less than 200 meters squared, -1 for larger. SWT-1 is right now in a draft. It's not been fully approved yet. But it raises the limit for small wind certification to 150 kilowatts. So, that will be a different context.

Now, each of these standards reference a whole family of standards for the different tests. And—yeah. And then more on the electrical safety side of things you have the UL standards, 6141 and 6142, and they reference—they're for the whole turbine system and they reference different standards. For instance, the alternator, I think, is 1004 and the inverter is 1741. And then you have the National Fire Association, which has got the National Electric Code.

Now, why so many? Are they really necessary? Unfortunately, the answer to the second question is yes. So, let's look at the they're necessary.

The National Electric Code covers pretty much everything electrical installed. And while there are different sections for specialty things like wind, the standard as a whole covers everything, and so the inspector doesn't have to have 26 different standards to evaluate your installation. They can look to see "OK, well this system is UL-certified so I don't need to look inside the system to see if they did that right. That's covered by the UL standard. All I gotta do is see if they've got the right circuit breakers, the right wire sizes, the right number of conductors in a conduit, things that are consistent across the whole range of technologies." So, the inspector doesn't have to become an expert in wind installations in order to inspect your installation.

Yeah. And then things like 1547, which is about how your generator connects to the grid, they're the same whether it's a diesel genset, a solar inverter, a wind inverter. The same set of rules applies and therefore you don't have to have 16 different rule sets for 16 different types of application.

Let's see. I think I've covered most of the rest of that.

So, how these weave together. An inverter is designed and tested to meet UL 1741. And it—UL 1741 refers to IEEE 1547 for power quality and protection and testing requirements. The IEEE 1547 is a—what we call a voluntary standard, but then when it gets adopted by UL as the basis for how they list inverters they make it required. And then your inverter has to be installed consistent with the National Electric Code. And part of the National Electric Code will say that it needs to be listed for the application. So, for instance, you may have an inverter that was developed for PV and it's listed for PV and you want to use it in your wind system. That would not be allowed if it isn't listed for a wind system. Or in some cases they're listed for any DC source, in which case then they're going to look and see if the DC that you're providing out of your wind controller meets the requirements for the inverter. In other words, is your maximum DC voltage within the range of the inverter it's listed for. Is the power range within the range that the turbine—or, inverter is listed for.

A listed wind system—so, where you have the whole system UL listed—will be listed usually with a specific inverter, or you may list it with a couple of inverter options. And if you for some reason have to change inverter suppliers you may not have to have the whole system reevaluated but you will have to add it to the listing by showing that it meets all the requirements that have already been established in the listing for the wind generator.

OK. So, the National Electric Code is—applies to anything electrical getting installed, with a couple of exceptions. Boats is not covered. RVs are not covered. And utility equipment, equipment owned and operated by a utility is not covered. And part of the way it ties everything together is that it requires things to be listed for the application. So, it ties back to the UL standards.

Now, the big part of the National Electric Code is chapters one through four. And those cover everything standard. So, wire sizes, terminator types, circuit protection, how many conductors you can have in a conduit of what size, how big the wires have to be for a certain current—and length, et cetera, et cetera. And then, there are other chapters that are special equipment applications. Chapter six is special equipment for independent power sources—like, wind electric systems is 694. 690 is PV systems. Special sections override the general sections. In other words, they contain exceptions. For example, the installation of a wind turbine for testing at a supervised site is permitted without all components to be listed. That was an exception. Another exception is that you don't have to ground the Wye of an alternator in a wind generator where the output of the alternator is getting rectified. That does not work. But otherwise, you would. In a normal situation a generator's center-tap Y configuration needs to get grounded.

So, it covers both the equipment and the methods and practices. So, again, the equipment must be listed and approved for the specific type of application that you're doing. It creates an option for field labeling in some cases. Although, you can't field label an inverter because they can't really test in the field for all the things that it needs to be able to do. But if you had a listed inverter and did not have a listed wind generator you could have a person who was licensed for this, usually someone from a national testing lab, come out and test to make sure that—and run some tests to make sure that it complied with the listing standards, even if it hadn't been—gone through laboratory testing for that. And that would actually need to be done for each installation. So, it would get expensive in the long term, but in the short term sometimes it's helpful. Also, it's used when you're—if you're doing large installations of stuff that's not listed, like a two-megawatt turbine are typically not listed because mostly they're applied to utility applications. If you're putting one in that isn't for a utility application you may need to get field labeling.

Labeling is part of the National Electric Code: what you have to label, where you have to label. Labeling to let first responders know that there's an independent power system here and where they have to go to turn it off. And then, as I've mentioned before, the size of the raceways or conduit, and how much stuff you can stuff in them, how high-temperature the wire has to be if you put more wire in a smaller conduit, et cetera.

Yeah. As we said, 690 is for PV systems, 694 is for wind systems. There's also a section for fuel cells and on and on and on. And then, in chapter seven, special conditions, you have something that does impact wind—distributed wind installation. Sometimes it's interconnected electrical power production equipment. You also have sessions on—or sections on storage systems, and a new thing in the last round was direct current microgrids.

So, the National Electric Code is updated every three years and it's based on having—anyone can send in a request for a change in the National Electric Code. They have to say exactly what they want changed and how they want it changed and they have to provide some rationale or substantiation of why. And then we look at every one. Now, for instance, I'm on code panel four, which covers all of the renewable energy equipment and a bunch of the stuff in chapter seven as well. And we may get 300, 400, 500 requests for changes and have to go through each of them and decide whether to use them or to use them partly and justify why or why not.

Male: Hey, Robert, it's Rob Wills.

Male: Oh, good.

Male: Hi. Just a quick comment. So, for folks watching, the next edition of the National Electric Code is 2022. Public input for this cycle has mainly ended and there are very few changes to 694. They're nearly all about dealing with definitions. So, a lot of the definitions in the individual articles are going to move to Article 100, "Definitions," just to centralize the whole definition thing. And the only substantive one that you might get a kick out of is that UL is asking that when we do a listing we say whether the unit is suitable for onshore or offshore. That's a new one. If you're looking at doing distributed wind offshore [inaudible due to laughter] being subject to certification.

Two things. One is even though public input is finished the panel, which is codenaming panel four, which covers 694 and these other articles, is allowed to make its own motions. So, if we had an input from one of you folks and said, "This is a real showstopper; it's something you really need to fix," we could bring it to the committee and we could probably get it put through. It's not a common practice but it's allowable that the committee can actually submit a change.

The other thing is a continuing debate as to whether large wind or large wind farms are other the NEC or not. Chapter 90, the beginning of the National Electric Code, section 90, says that the code covers pretty much everything apart from marine and utility-owned equipment. And by definition an IPP, an independent power producer, is also called an NUG, a non-utility generator, and so technically they can be interpreted to be under the NEC but they've largely escaped. For utility equipment, that's under the purvey of the IEEE National Electric Safety Code. And for anything that's not utility equipment—in front of the fence instead of behind the fence, as it's termed—then it's National Electric Code.

So, that distinction is still being worked out. So, 99 percent of cases for distributed wind were under the NEC. So, that's it for me. I represent the American Wind Energy Association on the National Electric Code and we're working on actually getting some DWEA representation there too. Thanks, Robert.

Male: Yep. Thank you. OK. A little bit about IEEE 1547, which is the standard for interconnecting to the grid. And just to delay any confusion, this is not what you get certified to or listed to. The UL standard is 1547—I'm sorry, 1741, but it refers to IEEE 1547 and includes the testing requirements, which is IEEE 1547.1.

So, that's been a huge change in that in the original inverter standard essentially the only thing that was required is that you give them good power, and that if there was anything going on on the grid that you tripped offline and got out of the way. In Germany at some point they realized they had sometimes 20 or 30 percent contribution from mostly solar at times, and that if they had an event that caused all of that to trip, the event would suddenly get very much worse. So, they spent several hundred million dollars reprogramming tens of thousands of larger solar inverters. And at the same time Hawaii and California, which in some places has also very high concentrations of mostly PV, realized that they were going to have—they were heading toward the same problem, and they developed standards to force the distributed energy sources to provide—to help support the grid in the case of an event, meaning you have to ride through, you have to provide reactive power or—there's a whole complex set of responses that you have to do to grid events. But mainly it's about staying there and helping support the transition while the grid responds and they bring more responses on and for those few seconds.

And that —

Moderator: Sorry, about ten minutes. About ten minutes.

Male: I'm sorry? OK, thank you.

Moderator: About ten minutes.

Male: I think I'm doing pretty good, then. But that made the amount of testing required in order to confirm that the inverter does everything it's supposed to increase exponentially.

Yeah. Right. This is more of the details of that.

Oh, the other requirement is communication. So, inverters now have to be able to communicate with the outside world. In many cases that's not being used, but it is going to be more and more into the future. And as I said, it—this all results in an exponential increase in testing effort.

Yeah, the only difference between UL 6142 and 6141 is whether or not personnel work inside the turbine enclosure.

Listing the wind generator as a system will include looking at each component and making sure that it is—if it's listed as a component, that it's being used inside the bounds of what it's listed for. And then, looking at—making sure there's no single-point failure that results in a catastrophic result.

I think I already covered field labeling pretty well.

A whole list of the sort of stuff, that documentation you need to bring to the national testing lab in order for them to begin working on evaluating whether your system is able to be listed.

And I think that's it. So, are we on track? Should we move on? Or do we have time for a question or two?

Moderator: We have time for questions. So, I don't see anything in the Q&A block. But if people have questions, either raise your hand, or at this point things are pretty good, so just unmute yourself and go ahead and ask your question.

And since we have a little time—I mean, not tons of time, but since we have a little time, if there are any questions that people have in regards to my presentation, we can certainly address those now. Or later.

No questions for Robert or Rob? Now's your opportunity. Two of the people that are writing those standards.

Male: OK, well, maybe we should take advantage. Oh —

[Crosstalk]

Moderator: Looks like someone's trying to talk but we're not getting any audio.

All righty. Why don't we move on, and if that person trying to answer the question, why don't you type it in and we can respond —

Male: I see a question typed in. Not sure I understand you right. "How do UL 6142 and 6141 or IEC 61400-2 connect to UL 1741?" [Laughs] Good question. OK.

UL 6142 and 6141 both are for the entire wind system, so one of the things they'll do is they'll make sure if you have an inverter that it already complies with 1741. And they'll also look at your alternator and see if it complies with UL—I think—1004. I may have that number wrong. There's a whole lot of numbers out there.

Male: That's correct. Yep. _____. Yep.

Male: OK. So, they'll look to see that everything in your system is listed and is being used within the bounds of what it's listed for. So, if it's listed for 600 volts and you're operating at 700, that won't fly. IEC 61400-2 is more performance, structural, that kind of analysis, whereas the UL is mostly electrical safety.

Male: That's good. So, embedded in 6141, 6142 is a pointer to 1741 that says your—in effect, your electrical system has to comply with 1741.

Male: Right. Yep. Yep. One more question here: "When you indicate a boat or utility for ownership, what is defined as a utility?" A utility—being off-grid is not a utility, unfortunately. I understand how it might show up that way. The utility means someone who generates power and sells it to other folks as a business. And there's more definition than that, but I'm not sure —

[Crosstalk]

Male: Yeah, control of a state entity like a public utilities commission, a PUC, would typically —

[Crosstalk]

Male: —regulated. There are also co-ops and munis and _____, but they're all utility. And I've got one more comment too, Robert, and that is the National Electric Code has been pretty stable for the past six, eight years. Robert and I have been working on it for quite a while, and our job to a large part is to make sure that the crazy suggestions don't get in that would be showstoppers for the industry. So, that oftentimes—showing what our work involves, it's like putting your finger in the dyke to make sure that the whole thing doesn't come apart, because you end up with a perhaps well-intentioned change that has consequences that folks don't fully understand.

Male: Yep. Or that has consequences in areas that they weren't even thinking about. When they proposed a change to things they aren't necessarily thinking about all the industry applications, et cetera.

Male: Yeah.

Male: Rob and Robert, looking back in the chat it looks like Michael Berdan was asking if there are currently any UL-listed wind inverters.

Male: No. But we're working on it.

Male: We're working on it, yeah. It's —

Male: [Laughs] Rob Wills just yesterday shipped a sample to Intertek for them to do the physical review. So, that's where it is for a 25-kilowatt, single-phase, 240-volt inverter.

Male: Yep.

Moderator: And then, before we move on quickly, and then just to note, there is an SBIR, small business research innovation grant, that was released by DOE to look at other wind-specific small inverters. So, that is out on the street. But we've identified this as a hole, a kind of complicated hole in the market for smaller-scale wind technology. And we should move on and have Brent provide us an update of SWT-1, or an overview of SWT-1 if there are no other questions.

Male: Yeah, and if you have questions later you're more than welcome to e-mail me or Ian or any number of other folks. And if we don't know the answer we can probably find out from someone we know who does.

Male: Yeah. One other thing to mention is that Ian Brownstein at XFlow is struggling to have an inverter for his product in the 40-kilowatt range, and he is talking about bringing together a consortium of manufacturers. And so, if you're looking for an inverter it's probably worth also talking to Ian at XFlow in Washington State.

Male: OK. Good session, guys. Thanks for all the electrical safety standards information.

Male: Simple. It'll be done tomorrow, Brent.

Male: Good.

[Laughter]

Male: Bye.

Male: Tomorrow. Tomorrow, is it?

Male: This is what we were worried about yesterday.

Male: Yesterday's gone. All right. Well, we're going to do it looks like one more session here, session three, before a break. I'm going to go over an update on AWEA SWT-1. So, thank you for being here, everyone. And my name is Brent Summerville and I am starting my second week at NREL. So, I'm the new guy. My background is manufacturing engineering and building and operating the Small Wind Certification Council since 2009. And grad school plus teaching at Appalachian State University, where I did lots of hands-on distributed wind and wind resource assessment work. But now I'm with NREL. I'm an NREL-ian. So, I'm excited to be here.

But standards. So, in my role as a certification body technical director/consultant I spend a lot of time in these standards. And you the audience as manufacturers of wind energy equipment certainly have probably spent some time reading, buying, and understanding these standards. But in many groups testing and certifying your product to these standards, and namely our national standard, what we're talking about there, the AWEA standard, has been identified as a barrier to market entry. It's time-consuming. It's expensive. It even scares people off a little bit from changing their products and improving their products because they're fearful of having to go back through this testing and certification process.

Now, this NREL CIP program is pretty awesome. The funding has helped reduce the financial barriers for those seeking testing certification of their products. But you still have to go through the process, which can be painful, time-consuming, and expensive for all of us, so there's an identified need to optimize the standards, which is a slow process and involves a lot of stakeholder coordination but is what we've been doing.

So, AWEA SWT-1, we worked on it in 2016, around that area, and we as stakeholders developed a draft. It didn't make it through the ANSI process in a clean way, so it was—after an ANSI audit of the AWEA Wind Technical Standards Committee processes it was remanded, the summer of 2019. And so, this really gave us an opportunity to step back and take another look at revising our national small wind turbine standard. We utilized that SWT-1 draft from 2016 as a starting point but looked at it again after a decade of lessons learned from using the existing AWEA standard, AWEA 9.1, which was released at the very end of 2009.

So, 2010, really a deluge of interest in testing and certification, and many, many people started and some have finished the complete process of testing and certifying a small wind turbine to the AWEA standard. In the entry slide you see the DWEA logo and the NREL logo. Ian and others from NREL—Jeroen, Robert—worked with Mike Bergey and DWEA to gather some folks together in person—remember that?—back in February of this year in Colorado to go through the AWEA standard and propose change, and then met again following the DWEA conference in Arlington, Virginia in February, and then many online meetings to finalize a draft, AWEA SWT-1, through those virtual meetings that you're all used to now.

But if you're not familiar with the AWEA standard, here's a quick overview. If you obtain the AWEA standard, as you know, it's not complete on its own. You can't do the work with just the AWEA standard in your hand. There are many national standards around the world. The Japanese have their own standard, which looks a lot like the AWEA standard because we play off each other, but they have their own Japanese requirements. The MCS program by the British has their own certification scheme. The Danish have their own scheme and special attention to micro wind turbines and whatnot. But we all lean on the family of standards from the International Electrotechnical Commission—the IEC—61400 standards for wind energy. And I've sat on these standards committees and it's challenging to get these standards right. All of you are involved in converting the kinetic energy in the wind to rotation and then relative motion of conductors and magnetic flux, but there—as you know, there are many ways to do that. And the—but if you want to test and certify your equipment we all have to play by the same rules as outlined in these standards.

The AWEA standard points you to the IEC standards as such. So, for power performance testing it directs you to 61400-12-1. And the latest version of SWT-1 points to the latest version of these standards for the power performance edition 2.

And in acoustic sound testing it points you to the -11 standard, which is now at edition 3. A safety and functions test is described in -2, edition 3 now. Strength and safety is a small amount of words but a big load of work where you do your design evaluation, model the loads on the turbine—no easy task—through either simplified load modeling or aeroelastic modeling, and then apply those loads to the major components and use mechanical engineering methods to determine the strength of those major components. And that's described in -2.

You have to pass a duration test per -2. And there's a new section called consumer labeling leaning heavily on the work from IEA Task 27, which resulted in annex on consumer labeling in the -2 standard. And then, a couple appendices: one on sound pressure levels at various observer locations and differing background noises. And then appendix B on conformity assessment. It is a bit odd to have conformity assessment in a technical standard, but all the conformity assessment, aka certification language, has been extracted from the main body of this standard and placed in appendix B. OK.

Scope. Now, specifically the scope has followed IEC 61400-2, which is up to 200 square meters of rotor-swept area, but this group says "You know what? The specific power is decreasing for all wind turbines, even small wind, driving the swept area up." And there's a gap of wind turbines between small wind that we're used to and to 50 kilowatts or so. There's a gap between small wind and large wind, multi-megawatt wind, such as the Northwind 100, the Pecos 85-kilowatt, the old endurance E-Series. Rotors were too big to be small wind but they were too small to be big wind. So, we decided to expand the scope to a peak power of 150 kilowatts and move away from rotor-swept area. Peak power seems to better define the maximum mechanical loads, electrical loads on the system. And so, yeah, this brings up—this increases the sizes of turbines that will be applicable to the AWEA standard. And you'll see in this presentation we set different requirements for different sizes within this 150-kilowatt peak power scope.

And I see there's stuff coming in the chat. "We need a micro wind category." Great. I'll talk about that.

And I'm sorry, when we get to acoustics I'll look at yours, Padma.

Definitions. We did change reference annual energy, which is a term aligned with the IEC standards. It's like "How much energy will this turbine produce in a radial distribution assuming 100 percent availability at sea level conditions with an average wind speed of—it used to be five meters per second?" We decided to move that to an average wind speed of six meters per second.

This industry is a bit conservative. We don't want to disappoint people, so we—we're presenting a reference annual energy at a modest and low five meters per second, where—and that's an indication of what the turbine might do at that location. But it was conservative. So, if we're going to compete with a decision on PV and/or distributed wind we thought it would be better—it would level the playing field to say, "Here's what the turbine can do in a six-meter-per-second wind regime" to better compare the energy output to the way PV is doing. And so, we hope to level that playing field right up front when people are comparing technologies.

And check it out: We do, Daryl, have a new definition here, micro wind turbines, and these are small wind turbines with a peak power up to one kilowatt.

Before I forget, how can we access these IEC standards? Well, they are available for purchase on the internet. The IEC web store, if you Google "IEC 61400-2" I think it's an IEC web store where you have to purchase these copyrighted IEC standards.

OK, power performance testing. Well, we cleaned up a little bit here. It points you to IEC 61400-12-1. There's a little bit of conflicting criteria on the wire run. We have said the wire run should be at least rotor diameters; IEC says the wire run should be—or, the connection to the load should be no further than three times tower height. So, what do you do? So, we said, well, we'll just go with the existing IED criteria that lives in that Annex H for small wind. And the same with database requirements. You're filling the bins. AWEA says keep going until you reach—until you fill each bin, each wind speed bin with 10 minutes and don't stop until you get to the wind speed which is 5 meters per second beyond the wind speed at which the turbine has reached 95 percent max power. And so, it's looking for that peak power, and so you might keep going until you find that. But IEC says just fill the bins from 1 meter per second below cut-in to 14 meters per second. So, we want to get this power performance test over and done, calculate the AEP, write the report, and so we've gone to the reduced IEC database requirements, and moved that peak power requirement to where we thought it belongs, to the safety and function test where you want to make sure you've collected enough power and wind speed data to demonstrate peak power of the turbine.

And the question, "Is that ten minutes per bin and one-minute averages?"—it is. It's ten minutes per bin and one-minute averages. So, ten minutes. I'm sorry—yeah, ten minutes, one-minute averages.

And a further savings is we say site calibration is not requirements. It's quite easy to fail or to not pass the terrain assessment and need to perform a site calibration where you put an anemometer where the wind turbine will be and derive these slow correction factors between the MET tower and the turbine tower, but looking back, the people that had to do that, those correction factors were quite close to one. In many cases the MET tower and the turbine tower are relatively close together for small wind, so we thought this would be a savings.

Acoustic sound testing is interesting. You've got to do dB math and it specifies equipment and it's a little weird. So—but we've got some great instructions on how to do it in IEC 61400-11, edition 3, which now has an Annex F for small wind. And the particular annex there does say that it's for turbines up to 100 kilowatts and we are proposing that turbines up to 150 kilowatts use that annex and be considered small wind.

We changed the uncertainty calculation to a "should" to save a little bit there. And acoustic sound testing is not required in our document for those micro wind turbines.

Strength and safety. Based on turbulence intensity studies through—from IEA Wind Task 27 and with measurements at typical installation sites the turbulence intensity 15 meters per second assumption was raised in this standard from 18 percent to 20 percent. And simplified load model is still in—you still—it still lives in -2, but we're limiting it to turbines up to 30 kilowatts and we're saying we're not recommending it be used for turbines with a peak power greater than 10 kilowatts.

The IEC classes, as you know, are I, II, III, or IV, from extremely windy to less windy, and then there's always been class S for special. We have decided to simplify things, that we limit this choice to class II or—and that makes a turbine pass those class II requirements. We thought that was suitable for installations in most locations, and there's still S for high or low wind, special wind regimes. And look, another exemption for micro wind turbines: design analysis not required.

Now the duration test. The duration test is a big chunk of the field testing and we knew that was a top barrier to innovation and market entry, so we spent time collecting duration test reports, selected 30 published duration test reports for small wind that were conducted. The tests were conducted from 2007 through 2018 over an 11 year period and they were performed to the -2 standard, the AWEA standard, the BWEA standard, and in one case -2—the new -2, edition 3. And the reports came from myriad test sites around the world, all excellent places, including Texas, Kansas, the UK, Colorado, Scotland, more Texas, Prince Edward Island, Spanish Fork, Utah, and of course Denmark.

Looking at some of this data, the vast majority of duration tests were per class II requirements—22 of 30. There were three attempts at a class I test but no one made it. They settled for a passing the class II duration test requirements. In one case the high wind requirement, 1.8 Vave, they made it to 23.7 hours. They needed 25 but threw in the towel and accepted a class II test. There was one attempt at a class III test that settled for a class IV. So, sometimes it's quite difficult to fulfill the requirements of the duration test, but as you can see, most—the vast majority fulfilled class II duration test requirements.

Now, we call this the six-month test, right? Like, "Oh, we've got to pass the six-month test." Well, guess what? That test can take 6 months but the average is almost 9 months, and some tests have gone on for 18.5 months. Why? Well, there are lots of requirements to meet. One of those is six months, and there are other high wind requirements and operational time fraction requirements. But one thing this study of duration tests is missing is down here at the bottom. We're missing a lot of dots, red dots, of turbines that were installed at a test site and failed early in the process and there was—there is no duration test report. So, I'm always trying to collect this but it's hard data to obtain. And so, as you know, some turbines, they fail early and so we don't know how long it takes to flesh that out. In some cases it doesn't take long. But these are the turbines that actually went through the full test and some actually still failed. Some tests were aborted. Constant problems. Some made it through six months and failed in post-test inspection. And some kept trying to achieve a 90 percent operational time fraction and never made it.

One of the requirements is you need 25 hours in wind greater than 15 meters per second, and in most cases people were close to that 25-hour line but in some cases went way above, up to 238 hours. Note there is one data point per the new edition of -2 which requires 10 minutes of 2.2 Vave. They obtained 610 minutes out of the required 10 minutes, but that was for a class III wind turbine.

Here is a high wind requirement they're trying to achieve in the testing: 25 hours in winds of 1.8 Vave. Vave is a factor of the IEC wind turbine class. But you're—you need 25 and in some cases that—the test kept going and the turbine achieved up to—over 600 hours of high winds and still failed because it had too much downtime and never achieved a 90 percent operational time fraction.

Operational time fractions are essentially uptime or availability, and a lot achieved 100 percent. Even some failures, pretty close to 100 percent but failed in post-test inspection. And some were right at 90 percent and then some failed because they never could keep going in order to get the operational time fraction up to 90 percent.

Now, another requirement is achieving at least 2500 hours of power production and a lot of people blew past that. Up to 6140 hours were collected in this test. Most tests achieved much greater than 2500 hours of power production as they were trying to meet these high wind requirements or get the operational time fraction up to 90 percent.

And FYI, the turbulence intensity as reported from these test sites, which are relatively smooth terrain, bumped up against 20 percent. And so, this, I think, is in support of raising the turbulence intensity at 15 meters per second from 18 to 20 percent for distributed wind turbines.

And what did these particular duration tests catch? What kind of failures did they catch? Inverter failures made the tests abort. Never could get it to work. And then in post-test inspection you see excessive friction, a locked-up rotor, inverter failures right at the end, a broken welds, loose nuts, tail damage, stress cracking, multiple failures of the systems, and then catastrophic blade failure not noticed until after six months post-test inspection.

So, upon further review these testings goes on and on forever and cost lots of money, and in reality the ones that really failed never even made it to the test report. So, the group says let's dramatically reduce the duration test to 10 hours in wind speeds of 15 meters per second and above, now based on those 1-minute averages like everything else, instead of 10-minute averages, and log 1000 hours of power production. And we think that should catch some of these early failures in the system.

Back to your favorite, micro wind turbines, Michael Berdan thinks that should be ten kilowatts and this group has proposed one kilowatt and below. And the micro wind turbines receive power performance tests, safety and function tests, that abbreviated duration test, but no acoustics, no design analysis, no blade testing. The cost of the turbine is relatively low, allowing taking some additional financial risk without some significant financial consequences. And validation of how the turbines are doing in the field will be done through—validation of the turbine will be through the testing and the post-testing follow-on field inspections we're about to talk about.

So, in Annex B, conformity assessment, we have—we decided that, OK, we've reduced the duration test requirements, which does catch problems, and so what we thought to counter that would be a more robust surveillance program. Now, if you certify a turbine there is a—there's already a mandatory surveillance program that has to be followed. And I would say A, C, and D are already in existence. You certify a turbine, you have some factory inspections to verify that you have quality assurance in the factory and you are building that turbine that was certified.

Let's go to c. You have to do annual reporting of all your design changes to the certification body, all the field failures, all the complaints, and the size of the fleet of certified turbines. And if you have significant design changes or safety-related field failures, those shall be reported to the certification body without delay. So, this—these systems are there.

We thought it would be more robust to have a new field inspection program. We realize that a turbine can be certified and still—still—have field issues, and we want there to be a robust communication conduit from the field back to the OEM and certification body so they can be understood and addressed. So, the sample size is five turbines in the field at different sites, as chosen by the OEM and the CB. If you only sold three then that's fine. Hopefully you sell at least five certified turbines and you find five that you can do annual inspections on by a competent third party such as the installer or service provider every year for three years. So, let's keep an eye on these turbines. Let's look at them closely for blade damage, for cracks, for degradation and significant wear of those major components and have that qualified personnel write an inspection report, send it back to the OEM and the certification body. And on that report we'll also want to just look at the annual energy production that year and an estimation of the annual average hub height wind speed.

And this is mostly to—this last requirement of reporting energy and wind speed is mostly to describe the conditions at the site where the field inspection is being performed. Did it turn out to be a three meter per second average annual wind site? So, you'll—which is saying you're not gonna learn much about high energetic wind at that site. So, it's a way to qualify or describe the conditions at the inspection site. But that is our proposal for a more robust surveillance program to keep a closer eye on the certified product once it's landed in the field, for all of our own benefits.

And boy, this might be too small, but we have a table in the standard now that says "Which category are you in: micro wind, 1 to 30 kilowatts, 30 to 65 kilowatts, 65 to 150 kilowatts?" If you land in one of these columns you go down and see what you—what some special arrangements are. Micro wind, you see you need duration test, power performance, safety and function, and consumer labeling. The 1 through 30, you can use the simplified load model, but if you're over 10 kilowatts it's not recommended but allowed all the way up to 30 kilowatts. And some details here on the validation of any aeroelastic models. And then down here on blade testing, static blade test required all the way until you get past 65 kilowatts, and then it is encouraged to do some accelerated fatigue testing for your blades, but not required.

OK. So, next steps on this standard. It's with the Wind Technical Standards Committee with AWEA. We've made it through public comment. We've made it through the committee comment period. The subcommittee has met. We've drafted comment resolutions. And we are now in the process of resolving those comments with the commenters. This effort is led by me, myself, Jeroen van Dam with NREL, and Mike Bergey of Bergey Windpower, along with a pretty substantial group in the subcommittee.

The goal here is to have an improved US national standard by the end of this year, and we are close. We hope that it will be considered by other countries. Mike Bergey presented it at the Danish International Distributed Wind Conference and there was some interest in collaborating. There—the holy grail is to just certify your turbine once and have everyone accept that. But at this point we're trying to lead the way with a national standard that is heavily optimized. And we hope this optimization, these changes to the IEC standards will influence a revision to these IEC standards, mainly -2, which will be starting up in the next couple of years.

All right. Well, thank you very much for enduring that. I'm gonna look back through some questions now, please.

[Crosstalk]

Moderator: Yeah, I think the one that was there that you haven't addressed already is the ten-minute bin, meaning ten one-minute averages.

Male: Correct. It's all one-minute averages now. And power performance of field bin is ten one-minute averages. And there's a lot of details in the standard about that but that's it: yeah, ten one-minute averages. Ten dots.

Moderator: Great. Are there other questions that people might have? We're a little bit—we're kind of eating into the break, but —

Male: Yeah, sorry about that.

Moderator: —any questions? One just from Padma.

Male: Yeah, OK, I'm going back to that now. Padma, the annex—OK, where are you? Lost it. The annex is—talks about different sound pressure levels at different observer locations with different background noises. But what was your question about? I lost it.

Moderator: Does—the Appendix A of SWT-1, do we need anything to translate IEC sound level to AWEA sound level?"

Male: Ah. Yeah. That—the details of how to calculate the AWEA sound level from the IEC sound power level, it is further explained with much more detail and clarification in the acoustic section of SWT-1. Basically, the AWEA standard says "What does it sound like to an observer 60 meters away from the sound source at a wind speed of 9.8 meters per second?" And so, there's an equation with details on how to do that. OK.

Moderator: Great. And there's also a question from Josh. "If the ITC is extended, will SWT-1 qualify for ITC credits?" And I can answer that kind of. So, the Distributed Wind Energy Association has sent a letter to the IRS telling that the standard is being updated. So—and DOE will do the same once the standard is in place. Generally speaking, in previous history once the new standard is in place that supersedes an old standard they do, but we haven't—no one has received anything from the IRS saying such, and obviously no one has exercised it to approve it. But the general assumption is, yes, it would.

Male: Yep. "Does Europe use IEC standards?" Yeah, Europe uses IEC standards. The last thing to do when you develop one is translate the whole thing in France. And yeah, it's kind of a Eurocentric standard suite, so yes.

And Tim Olsen does point out that micro turbines can still harm someone. But you know what? Without a current pathway certification is very rare for micro wind turbines to go through this process, so we're hoping they will, and they will achieve a safety and function test, power performance test, duration test. So, we're helping they will, Tim, instead of not.

"Peak power is defined—peak power—how is peak power defined?" Well, it's the top of the power curve, so it's the then—the maximum then-averaged power output. So, if you've got a scatter that goes way above and the maximum is way below and the minimums—and a big cloud around all those means, but it's the maximum binned power. Peak of the power curve.

"Many turbines in the market are not certified through these standards. What do they say are the challenge or consequences?" Well, if they don't—if they're not certified they're not eligible for incentives and they're not on that list of eligible turbines for incentives. So, that's the main thing. I would say that is the main consequence.

OK. I don't see any more questions, so thanks, everyone, very much for your attention.

Male: The other thing is that if they're not certified or listed or _____, when you can get them connected and installed, why, your inspector may come around and say, "No, I won't accept this" and not allow you to connect it.

Moderator: Great. Thank you to —

[Crosstalk]

Moderator: If people have kind of additional questions, definitely type them into either the Q&A or —

[Crosstalk]

Male: Thank you all very much.

Moderator: If you have additional questions, please put them into either the Q&A or chat, though everybody is really using chat at this point, which is fine. So, we now have a break, basically a lunch break for people on the East Coast. And so, we'll plan to come back in ten—or, sorry—at 11:30 our time here in Colorado, or 1:30 on the East Coast. And so, please take time. We'll leave the lines open and stuff like that, so feel free to kind of mute yourself and stay on this line if you would like to. You're also, if you would like, happy to set up your own breakout. So you can break out, set up a breakout if there's someone here that you would like to talk to during this break session. So, go ahead under that breakout and then just create your own breakout session, and you can do that.

So, without further ado, thanks, everybody. We'll be kind of being quiet for the next 30 minutes and we'll see you all at 11:30 local time.

[Silence from 2:00:00 to 2:26:00]

Moderator: Hello, everybody. We'll be joining back up in just a couple of minutes, jumping in with Dean Davis from Windward Engineering talking about test sites. So, finish filling up your coffee mug and we will be back in a second.

Dean, are you back?

Male: I am.

Moderator: Great. Why don't we—Rachel, why don't you give—yep, exactly. Reading my mind. Thank you so much. Always—it's always a pleasure when people do that. Good.

Male: One step ahead.

Moderator: We'll give folks a minute or two and then we'll go forward.

Male: OK. My mic sounds fine?

Moderator: Yeah, no, you sound great.

Male: OK.

Moderator: You sound great. We don't see your picture, so—camera—you could do that. Yeah, you're —

[Silence from 2:27:37 to 2:27:59]

Moderator: All right. Why don't we get going? We're a minute past. We don't want to get too far behind again. So, thanks, everybody, for joining us here. We're going to kind of continue with the—with our dialogue, a quick conversation with Dean talking about test sites. And then, again, we'll go into this breakout session with the four different groups, and that will allow you to kind of dig into a little bit more of the meat in regards to the different topical areas within the CIP effort. So, that should allow some pretty good detailed conversations. We're going to that coming up.

So, without further ado, Dean Davis is known across the industry as well as the test site at Spanish Fork, so it's great to have Dean here again to give us kind of a brief overview of what it takes to actually test a wind turbine and collect the data that can be useful for certification agencies but then also to be able to make really good decisions about what you do. So, Dean, without further ado.

Male: OK. Thanks, Ian. Good afternoon, everyone. So, again, I'm Dean Davis from Windward Engineering. And so, just a quick background. I won't spend a lot of time. I've been at this for quite a while. That's a picture of me back in 1990. I went to Cal Poly for a senior project. I basically designed, built, and tested a small wind turbine. I came out to Utah to study under Dr. Hanson doing some graduate work, really around computer modeling. Started the test site at Spanish Fork in 1994, really to validate our models. Just one test led to another, and we've tested over a dozen different wind turbines. '06, started a company called Endurance Windpower with a few other people and took a product to market. Had the trials and tribulations of small wind, distributed wind. That company went bankrupt in 2016 and now I'm back really in the consulting world with Windward Engineering, really around that computer modeling and testing.

So, as you've seen in other slides these are the categories for the CIP that have been offered in the past. And I'm highlighting three of these here: so, prototype testing, certification testing, and type certification. So, I'm going to focus on those, basically around field testing for the most part.

Prototype testing, just to kind of get into a little more detail here, prototype testing is what it sounds like. You've got a turbine designed and you aren't ready for certification but you are ready to really see what it does in the real world in terms of field testing. Certification testing and type certification are really just the same thing. You're trying to get your turbine certified and it really is—the distinction is the size of it. So, less than 200 square meters, greater than 200 square meters. You can—you'll find in this presentation I'm really going off the old AWEA standard and the IEC standards. The IEC standards aren't going to change. Brent showed us where—and you'll see in my presentations where I think what Brent has been doing and the work of that group has been really trying to simplify some of the pain points within certification, in particular for the—well, these smaller turbines, which used to be classified as less than 200 square meters. Maybe it will be classified less than 150 kilowatts.

Probably a lot of you would know, this scatter plot over here really just shows kind of where that breakdown is in terms of size. If you're less than 30 kilowatts, more likely you're in the certification testing unless you had a huge rotor on it. If you're greater than 80 kilowatts, you're in the type certification—again, unless you have a tiny rotor on it. And then, right in that middle range, 50 kilowatts, you could kind of go either way. There will be designs that are greater than 200 square meters, there will be designs for less than 200 square meters.

Down in this table I'm just kind of showing—this is what NREL offered for 2020 in terms of funding these tests. So, prototype testing was up to 250 kilowatts—or, $250,000.00. Certification testing, $170,000.00. And type certification, $800,000.00. With a price participation of a minimum of 20 percent, you could see these total costs are going to be somewhere between $310,000.00—or, $200,000.00—a little bit more—up to $1 million.

So, to—what are the pieces of these certification tests? So, for the certification you've got a design evaluation and then you've got these four field tests that are required. So, you have power performance, acoustic, safety and function, and duration. Again, Brent kind of touched on these, and duration as an example; AWEA is trying to make that a little easier, less onerous on the manufacturer.

Type certification, that's a big project. There's not many ways around it. You've got the same thing: You've got a design evaluation. A little more detailed: There's a design basis evaluation, then a design evaluation. Again, four required tests, although they're different. You have power performance, safety and function. You have a mechanical load and blade structural, with acoustics being optional. There's an added manufacturing evaluation and then our summary, final evaluation.

The standards really are what we're been talking about, these IEC 61400 standards. Someone was asking earlier where do you find them? You can just hop on the internet and type in these IEC 61400-… if you're interested in acoustics, -11, and it'll take you to that web store and you can buy it. I poked online real quick and it's somewhere in the $300.00 to $700.00 range. Each one has a different price. The ones you're gonna run into—and if you were to get a CIP project you're going to be dealing with power performance, acoustics, the -1 or the -2, which are the safety standards, the AWEA standard for small turbines. You might be thinking about mechanical loads; that would be -13. You can see here. And the standards go on and on. You can have gearbox standards and that kind of stuff. But primarily, those are the ones you're going to be involved with.

So, in this process of certification let's make the assumption that you're a manufacturer and you're going to—at that point you're going to want to kind of pitch yourself with a certification body, because you're really going to go through this process as a team. And I've thrown up here the most typical certification bodies that I know of for North America: Intertek, SWCC, UL, TUV NEL, DNV-GL. There's others. You're going to be working with them in a sense. And you may be doing self-testing, which is allowed on the smaller turbines. Actually, I think it's allowed—I'm not sure on the larger turbines. But you can do self-testing or you can also be working with a testing organization. So, you could hire Windward Engineering or Renewtech or some other organization to do your testing. But you are going to be working with a certification body as well. And the certification body is actually the one—so, when I do a test I'm not doing the certification; I'm doing the test. But the end result of my work will be reports, and those reports will go to a certification body and they're the ones who will actually do the certification.

I threw up here some existing test facilities that I'm familiar with. Intertek has one in New York. Appalachian State, which Brent was associated with—I guess still is—in North Carolina. Renew Test in Texas. UL is in Texas. The National Wind Technology Center in Colorado has done numerous tests. And Spanish Forks. I actually saw someone on the chat mention that they've got a test facility in Iowa as well, which isn't on this map, but looking to chat if you're interested in that.

So, for the certification body to do a certification, they obviously are going to have to have confidence that the tests were done in accordance to quality assurance procedures, in accordance to the standards and that kind of stuff. So, there's going to be some due diligence around that. And the amount of due diligence really is a function of who's doing the test. So, if you go to an accredited laboratory I think that due diligence is already taken care of, really. If it's not an accredited laboratory—which a lot of test sites won't be accredited, and if you're doing a self-test you wouldn't be accredited—then there will be a process of going through this due diligence, where you look at—they're going to want to have confidence in the quality assurance, the testing procedures. They're going to interview personnel. They're going to want to give you some data for you to post-process and make sure you come up with the right answer according to the standards. They're going to want to visit the test site.

And in the end they're just—if they're going to put their stamp of approval on it they need to be able to trust the data that's—the data and the reports that they're seeing, that are being submitted.

So, in this whole process obviously you need a test site to do field testing. So, I thought one way to look at it is what would be an ideal test site? And I'm not—it's probably pretty hard to find an ideal test site, but if you could, it would have regular, non-seasonal winds. Kind of simple. You don't want to put up a wind turbine and sit in a doldrum period and not be collecting data. You need to have favorable zoning and permitting laws; you've got to get a turbine up. And to be honest, it's probably beneficial if it's not in the middle of nowhere where you literally have to drive three hours to get a bolt or something like that, because you're just going to need stuff like that.

To do a power performance test, ideally you have an unobstructed space. Just flat ground. No vegetation. No obstructions. You do need occasional high winds. If you had a test site that always blew nice and consistent but never blew hard you're going to have a hard time filling those high wind bins.

Acoustics. A little different parameters. Obviously, you want a quiet site. You don't want a lot of background noise. We're right next to a highway; that presents a challenge. I know NREL has a concrete plant nearby; that presents a challenge. It can—birds—lots of things can present challenges. Airplanes. Unidirectional winds would be ideal—not necessary but ideal—meaning the microphone has to be somewhat in the downwind direction, so if the winds are moving around all the time you're constantly chasing with the microphone. No nearby reflective surfaces. And then the ground cover is unchanging. It presents a problem if you're got high grass and then low grass or snow or north snow—that can change the acoustics of the background.

Duration. The key here is you need a lot of power-producing winds. You don't need extreme winds. You don't need to get 30 meters per second kind of winds. But you do need high winds to fill those upper bins. As Brent mentioned, the next version of the AWEA standard is trying to address that, to make it not so onerous to get those high winds. And then, I just threw in I think some variability in the ambient temperature is a good thing. It's not necessary to do a duration test, but I do think if you've testing in Florida, the first time that turbine is installed in Montana there might be surprises. So…

For the loads you do want occasional high winds. You want to see what happens, what the load, how the load changes with high winds. I said regular times of calm. It's more labor-intensive, the load measurements. You're going to be more often debugging instrumentation, calibrating instrumentation, adding instrumentation, that kind of stuff. It's nice to not be doing that in windy conditions. You do need variability in turbulence to fill the bins that you're looking for.

So, I just said here the ideal test site may not exist but most shortfalls can be mitigated with ingenuity, time, and money.

So, here's just an example power curve. And really, this—I'm not going to address anything in particular with this slide, other than I'm gonna spend the next ten slides or so kind of getting into detail on the power performance. And I spent a lot of time on that one relative to the acoustics or duration or some of the other tests because I feel like that's the foundation test, the test that if you're doing certification you're going to be doing it, but if you're doing prototype testing you're almost certainly going to be doing it. And I think in a lot of ways it's the most stringent on requirements of how you place your anemometers, where is the secondary anemometer, what instrumentation you use, and that kind of stuff. So, my feeling has always been set your test up really around power performance because that's one you're going to do. And then when you do duration you can use the same anemometer. It's higher quality than is probably required. It's more stringent in its location, but if you set it up for power performance then all the other tests will kind of follow out of that. You're adding new pieces for each other test. So, the acoustics, obviously, you add recordings to the test. But all that other data, the wind speed, RPM, turbine status, all those other things that you had used for power performance now are in place and ready to go for acoustics.

So, when starting a power performance there's a lot of steps, and I'm just going to kind of step through it, but first would be a site evaluation. So, here's a picture of a site that would easily pass our requirements. And so, this would just be—show on paper that the site is flat, it doesn’t have obstructions, it doesn't have nearby turbines, et cetera, et cetera. You can see in this image that would be easy to do, and that would just go into your report and you're done a site evaluation.

Most sites will have a bit of analysis. You'll actually look at "Is that an obstacle? Is it close enough? Is it large enough? Is it tall enough to be considered an obstacle?" So, the standard lays out all these parameters on what an obstacle is, what a nearby turbine is, et cetera. So, most sites are going to have something to analyze. But again, you can on paper analyze it and say it's acceptable and tolerable to the standard and it just goes into the report.

Other sites like ours, it's a little more challenging. And so, we have typically done a site calibration. I'll kind of run through that, what a site calibration looks like.

So, again, here's our test facility in the foreground. It's tight. It's compact. It's not a lot of land. We've got a lot of turbines, a lot of things involved in there. We've got a wind farm up-canyon. Actually, those turbines are large enough that they present themselves as obstacles to our test facility, so that goes into our report. But if we're looking at this test turbine here in the lower right—let's see. So, we'll need a MET tower. And in general you're going to want a MET tower—it doesn't have to be upwind, but generally you don't—you clearly don't want it downwind. You want it off to the side or in general upwind, two to four rotor diameters away. I highlighted that in _____.

You're going to look at nearby turbines. You're going to analyze whether those are—did that—are they considered nearby turbines in accordance to the standard.

Nearby obstacles. We have buildings. We have storage containers and trailers and that kind of stuff, so we analyze those.

And then we look at the flatness, and we have this railroad path adjacent to the test facility. So, that—all of those are factors on a site evaluation and all of that is outlined in the standard, the power performance standard.

In the end it's easier for us to do a site calibration but I'm not saying a site calibration is easy. It can be challenging. And again, Brent and his group is looking to say, "Yeah, maybe that's not necessary for these smaller turbines." So—which would be a good thing. It's just—that's their goal, is to reduce pain points. But for us, I'll go through it real quick on a site calibration.

In its essence, really what you're doing is you're putting your MET tower up, the one you're going to actually use for power performance. And then, ideally, if you don't have your turbine installed you're putting another tower at the turbine location and you're putting an anemometer at hub height, and then you're just going to start collecting data. And the requirements—all, again, all laid out in the standards—is that you would bin the data into 10 degree wind direction sectors, and then for each of those 10 degree wind direction sectors you'd want to have a minimum of 24 hours of data. You want some high winds, some low winds. And then you just want to see that they converge. As you plot it out, as you get more and more data, you'll recognize that, oh yeah, on average the difference between the MET tower and the anemometer at the turbine hub height is two percent difference. And it really does kind of converge into the two percent difference. And that's essentially—and then, once you do your power performance you're going to use that peak, that 2 percent or 1.6 percent difference to basically correct the wind speed you're collecting at the MET tower.

So, here's a quick example for a site calibration for us. We have it easy in a sense that our site really is unidirectional. Our winds essentially come from one—or, from a 30 degree pie. And you can see for us, again, our primary wind direction is 160 degrees. We get some winds at 150; we get some winds at 170. But we just really need to calibrate those three sectors in our situation, and that captures—that doesn't capture all our wind. We're going to have throw some wind out when it's outside of that wind—out of those sectors. But in general for us it's an easier way of dealing with the fact that our site is difficult to evaluate. But if you have a wind growth that looks something more like this, that's going to be a harder site to operate. There's a lot there. Some of those winds might only come in the spring, which could be a challenge if you're trying to calibrate it in the summer or the fall type of thing.

So, anyway, site calibration can be a challenge. Ideally, you want to basically on paper show that you can pass the requirements of the standard.

So, now you've got a site. You've done a site calibration—or a site evaluation or calibration. Now it's time to get some instrumentation—I guess to do a site calibration you've already got some of this instrumentation. But just running through the MET tower, obviously you need a tower. I've used these freestanding and I've also used guyed towers. You're going to basically have a primary anemometer at hub height; you're going to have a secondary anemometer below it. You're going to have wind direction below it. You're going to have air temperature and air pressure. And again, the standard lays all of this out in detail of how far from the tower should that secondary anemometer be? How low should it be compared to the primary anemometer, et cetera, et cetera? So—and once you get into the standard it really spends a lot of time giving you the details of—even to the extent of how vertical does that primary anemometer need to be? It needs to be within two degrees of vertical.

You're going to have more than likely some kind of instrumentation shed where you're going to have some other instrumentation—or you're going to bring all the instrumentation to that shed. This is an example of one that I've done. The power transducer's in the shed, the power supplies to run the instrumentation is in the shed, and the data acquisition system is in the shed. I've used—I have been for many years—I've been using LabVIEW National Instruments for data acquisition. It's worked out well. That hooks right to a computer and you actually log the data on the computer. You don't have to use that; there's many data loggers that would work just fine. Campbell Scientific is a popular one.

There are other pieces of instrumentation that you might want to add or that are recommended. The standard actually recommends that you measure RPM and turbine status, but again, you're spending a lot of effort to do this test, and sometimes it just makes sense to add a few more channels of data, which may help your engineers or your control guys to really evaluate something. So, pitch angle, wetness, rotor azimuth, et cetera, those kind of things. The picture there is just an example, a relatively inexpensive—that's a low-speed shaft, the back end of a low-speed shaft on a wind turbine. I've got an encoder there that's going to give me pulses that I can read for RPM. There's another instrument that those wires are going into. That's like a potentiometer kind of thing, so that's actually giving me azimuth angle. And I've really enjoyed the 3D printing world because it just allows you to make these custom mounts and instrumentation stuff that fits perfectly to—this one-off instrument mount that you need.

I'm not going to spend a lot of detail here but I just wanted to run a table that gives you some ballpark idea of what some of this instrumentation costs. You can see rotor speed, I just have a an encoder—relatively inexpensive. Power transducer—I've found it's usually about $1,000.00 to $1,500.00. Again, the standard has requirements on the quality of this instrumentation, so you'll need to make sure that it's of this quality or higher. Anemometer—these are pretty high quality anemometers, $1,500.00 apiece. So—and in the end I came up with a total, about $24,000.00 that you're going to spend on instrumentation.

Obviously, if you work with a test laboratory they may have this instrumentation. They may just rent it to you. There may be ways of lowering this cost. If you're doing it yourself you're probably going to need to go out and get this instrumentation.

So, this instrumentation, when you get it, it's going to have calibration sheets associated with it. And that's a requirement. You're going to need these calibration sheets. It's going to have scales and offsets that will float into your data acquisition system so that you can convert milliamps or volts or R2Ds, any signal resistance, any signal into engineering data—kilowatts, degrees, that kind of stuff.

So, calibration sheets are important. They'll end up—copies of them will end up in the report, the final report. The uncertainty analysis relies on these calibration sheets as well. And you will want to check with your certification body. We're run into situations where some certification bodies require calibrations that are ISO 17025 accredited. So, that—sometimes that's standard on piece of equipment; other times that's an extra cost.

So, now we're all set up. We've got our turbine. We've got our instrumentation. We've got our MET tower. We've got our data position. We're starting to collect data. What's required? How much data do you need? Obviously, you're going to need a certain amount of data and you're going to need a certain amount of the right amount the data. So, what is the minimum requirements?

In terms of the IEC you're going to bin your data into half a meter per second bins and you're going to want 30 minutes of data in each bin. And you're going to want to go one meter per second below cut-in and then one and a half times… these parameters. One and half times the wind speed where 85 percent of rated power is reached. In the example I'm showing here that would—for a 5-kilowatt turbine that reaches 85 percent of that 5 kilowatts at—which is 4.25 kilowatts, it reaches that at 10 meters per second, you're going to need to go to 15 meters per second. So, in this situation you're going to want to fill each half-a-meter-per-second bin between 1 meter per second and 15 meters per second.

And then overall you need a certain amount of data. So, you need 180 hours of data. So, we added a small requirement that Brent mentioned, which—that's probably going to change, but it can add it in this example and it ends up—you have to get a little more data. You have to—instead of going out to15 meters per second you actually have to go out to 16 meters per second.

So, how long does it take? Obviously, that depends on a lot of factors. You're gonna—how much data is in your valid wind direction sectors versus how much data is outside of that, because when it's outside you're not going to be using it. A lot of times you're waiting around for those high winds, trying to get 16 meters per second, especially if it's summertime. You might be waiting around just trying to get some kind of thunderstorm or something that will get you to fill those higher bins.

And then I said here, at least according to the old standards, if you've got a turbine that hits rated power at very high winds, that pushes out that maximum wind speed you're going to need to get, so that can be challenging. I think the other thing that can be challenging is a turbine on a real short tower. Again, that will cut down your higher winds and that can factor into it.

I have here NREL suggests three to six months. The beauty is you can perform a power performance along with a duration test. So, in some ways the duration test is going to dictate the total length because that's six months or longer. You can a lot of times just fit your power performance within that. Actually, you can—often I fit in a couple of power performances within a duration test.

So, now you've got your data. You've got enough data. And to be honest, really, the sequence of this is you're going to want to have your post-processing set up as you're collecting the data. So, you're going to basically be—you're going to have your spreadsheet set up, as an example, that has all the parameters and equations from the standard to do it, to do the post-processing and you're just going to be feeding it data as—you're going to just be feeding it data as the data comes in.

So, as I mentioned here, I do it all in Excel and I think that's a reasonable way to do it. Excel is really made for this type of work. The standard uses clear and well-defined equations. So, when you get into the standard I don't think there's anything real tricky in the power performance. You're just going to basically program or write in your equations in Excel according to the standard. There's lots of details on the standard, especially when you get into the uncertainty aspect. I think you need some kind of round robin due diligence. You need some other person looking at it and picking up errors and problems and that kind of stuff. But overall it's a doable process.

I say here the uncertainty portion is not trivial. There's old NREL reports, power performance reports, which they go through the uncertainty analysis, which is a good reference. And then there's also some presentations that they did that are helpful in that regard.

When it comes to reporting—which that's the goal of this test; you want to come out with this report—is, again, the standards are very detailed. It tells you exactly what it wants, it tells you what plots, it tells you what tables should be in the report. So, really, you just look to the standard as a guide. Again, there's old power performance reports that NREL has done. Those are great templates to look at. Just an example: Our reports, I looked at a few, they're about 90 pages in length, but a lot of that—I say here 55 is appendices. So, that's the site calibration for us; that's the calibration sheets, et cetera. NREL reports tend to be a little shorter. When I looked at that I got about 50 pages in total length, but again, half of that is calibration sheets and that kind of stuff.

So, acoustic testing. So, that really kind of runs the power performance from start to beginning, from a site evaluation all the way to the report. The acoustics, I say here the good news is that lots of the instrumentation is in place at this point. The times are much shorter as well. The bad news is there's significant challenges in the data collection. The data post-processing is much, much more challenging. And then, you do need some new instrumentation. You need some—you need a microphone, you need a calibrator, you need some software to basically—for us, we use Noiselab. That actually collects the data, it does the calibrations for us, as well as turning recordings into dB measurements. So, you do need to pick up a little more equipment.

In terms of what's required, how much data is required, I've got two slides, one if it's greater than 100 kilowatts, one if it's less than 100 kilowatts. I'm going to assume most people are more interested in the less-than-100-kilowatt version, and for time I'll just skip to that one. So, that all falls out of this Annex F of the acoustic standard. You're going to want—there is specifies you're going to want wind speeds ranging from cut-in to 11 meters per second, and those are one-meter-per-second _____. It says if possible the cut-out, especially for speed control mechanisms. So, ideally, if you've got —

[Crosstalk]

Moderator: Dean, you're starting to break up.

Male: Am I? OK. I don't know why. Let me turn off my video. Is that any better? Hello? Hello?

Moderator: That's better.

Male: You can hear me OK?

Female: That sounds better.

Male: Oh, sorry about that. OK, well, finally —

Female: Yeah, sorry about that, Dean. Thank you.

Male: Yeah, you guys are breaking up too. Step in again if you can't hear me. I'll assume if I don't hear anything that you guys are hearing me.

So, I was saying —

[Crosstalk]

Male: OK. you want to go from—yeah, I had cut-in to 11 meters per second, but even higher, especially cut-out, I think that's important for furling turbines, fluttering turbines, things that are real dynamic like that. It kind of makes sense. That's where the acoustics are probably going to get noisy.

The measurements are one second—or ten seconds, sorry. So, they're really short little recordings in a sense. And at each wind speed, then, you're looking for 12 measurements with the turbine operating and 12 ideally with similar backgrounds. So, ideally you could measure—you get your 12 measurements with the turbine, you stop the turbine, you get another 12 just with the background. That's the ideal setting. As you can see, that's only two minutes for each bin, so it's not a ton of data. But it can be challenging to get. There's just lots of reasons that it's challenging. But in the end I find it's likely weeks, not months. I think the longest acoustic for me was kind of in the neighborhood of two weeks, maybe three weeks.

The microphone does need to be roughly in the downwind direction from the turbine. So, if your wind direction is moving all around you might need to be moving the microphone around and kind of chasing the wind turbine. Measurement chain needs to be calibrated before and after each measurement. Ideally, you get the background in the same condition. So, we've had situations where I captured the turbine operating at ten meters per second when there wasn't snow on the ground and I got a background when there was snow on the ground, and that's a little challenging. Ideally, you want it to be the same.

And then, probably the biggest challenge is it has to be quiet. And it's people, it's traffic, it's birds, it's planes, it's grass, it's crickets, et cetera. It just goes on and on. And it's amazing when you're doing an acoustic test how—what a noisy world we live in. You start hearing everything. From my site I've got a pole—it's not even a wind turbine-related thing, but it's got a wire that's slapping. These vents in this picture slap in the wind, so for an acoustic test I'll duct tape closed. So, that's definitely part of the challenge.

On the post-processing, again, you'll want some software—we use Noiselab—where you're going to—Noiselab, like I said, is actually going to do the calibration for us, do the data collection, as well as turning the recordings into dB numbers that we can export into Excel. The rest of it we do with Excel, which works OK. It—again, it's a lot for Excel. Brent mentioned that there might be—well, there's a move towards not doing the uncertainty, and that's probably 40 percent of the data analysis. So, not having to do uncertainty definitely simplifies things.

As with power performance, the standard uses clear and well-defined equations, but they are complicated equations. There's a lot of variables and there's a lot of equations. So—and I think the other thing that makes it a little challenging is I don't think most of us are real good at intuitively knowing dBs, so it's like if your answer says 48 dB versus 49 it's not clear-cut where it's like, "Ooh, there's a problem" type of thing. So, that adds to the challenge that most of us aren't acoustic engineers.

I say here data analysis absolutely needs to be validated. So, when I put the spreadsheets together I had another engineer that was basically doing the exact same thing, and then we're just saying "What did you get?" "And what did you get?" And we're comparing and it's like "That's not what I got" and then we're going back and looking at the equation and analyzing the standard and trying to decide who's right, who got it wrong—sometimes we both got it wrong—type of thing.

Tonality is laborious. It requires lots of manual clipping, at least the way I've done it. I think Noiselab could be trying to put together software that will help out in that regard. Again, that's one thing that's being proposed to not be necessary in the new release standard. And then, overall I just said significantly more difficult that power performance in terms of data processing.

Duration tests. This only applies to the smaller wind turbines—so, less than 200 square meters. Purpose is you want to—one purpose, I guess, would be looking for infant mortality. You're trying to see if this thing can last at least a number of months. Structural integrity. Material and performance degradation. Quality of environmental protection. Dynamic behavior. That kind of stuff. Like I said, in some ways it's an infant mortality test, or if it lasts long enough, what are the other areas that probably need to be addressed that are starting to show weakness?

Unlike power performance and acoustic where you're presenting data, the duration test is a little different in the sense that it is kind of a pass/fail test. So, you just try and check all the boxes, and then if you do you pass, and if you don't you fail. But you need reliable operation, greater than 90 percent availability. In the current standard you need 6 months of operation, 2500 hours of power production, and then these smaller time frames, 250 hours and 25 hours—these are really around high winds, depending on what wind class you're at. And then AWEA adds a little bit extra: 24 hours, again, the way it stands right now, that are greater than 15 meters per second.

And if you have a major failure—and again, your certification body will help you to evaluate whether it's a major failure—if it's a minor failure you can fix the problem and probably keep going. If it's a major failure you're going to need to restart the test. If you do have a significant downtime you can lengthen the test, and if you have 100 percent availability for that extra time that you lengthened it you'll bring up your average availability and you might get it above that 90 percent. So, it would be an option of fixing a problem and continuing. If it's a significant downtime it might make sense to just fix it and start over type of thing.

Safety and function. Kind of the way it sounds. You're really trying to just evaluate that the turbine is safe to operate and that it functions as designed. So, in a lot of ways it's an extended commissioning test, but you do have data, you have—you've got wind speed, you've got some other measurements. You've got power, the power transducers and all that kind of stuff. So, you're basically just going to a commissioning test, an extended commissioning test with a little more detail where there's data available that you can put into the report to show kind of how the turbine operated in accordance to its design. So, what happens with a grid fault? If you were to purposefully remove a sensor, a wire, cut a wire in the control system, does the turbine safely shut down versus get into an overspeed or something like that? And then, you might want specific events, like a high wind start-up would be a good one. A high wind shutdown. Those kind of events.

Mechanical loads. Again—I guess I'm running a little short on time, but in terms of mechanical loads, there is a standard, -13, that deals with this, goes through all the details. It's not required in a certification test. It may be useful to have some loads but you're not going to need a full mechanical loads kind of test process or program. Again, -13 really defines it; it talks about the minimum required channels. It has recommended additional channels. It tells you what loads you want to see—braking, high wind shut down, grid loss, start-up. It wants to see normal operation in various wind conditions with various turbulence. And then it really talks about how to post-process the data and present the data.

So, in summary, kind of trying to wrap it all up and give some guidance on rough timelines and rough costs, we kind of go through here. But setting up your test, a few months, $20,000.00 to $40,000.00. Power performance, three to six months, another $30K to collect the data, process the data, and write the report. Acoustics, you'll see it's actually more expensive than power performance, because I feel like the post-processing is just labor-intensive. There's a lot there. And the report is actually a little more complicated, I believe. Safety and function, $15K. Duration test, that's just time collecting data, writing the report, processing the data. Mechanical loads, that's an expensive one if you're really going to try and do it in accordance with that -13. There's a lot of channels, a lot of work involved. Blade testing. I've never done a blade structural test but I have a quote for it. It came in at $235,000.00 and it was a six-month test, so pretty expensive there. And then, a manufacturing evaluation—again, something I don't have the expertise to do, I've never done it, but I did get a quote one time for $65,000.00.

So, in summary, certification testing, I've got the field testing gadding up to $175,000.00, working with the certification body at $35,000.00, so right around $210,000.00. Type certification, you'll see here much more expensive. Field testing, it's got that mechanical loads, tests. That's going to add up—I guess actually I put that into "other"—no, "other tests" was the blade and the manufacturing, And then certification body, they were much more engaged, a lot more work—$900,000.00. And when I started—these numbers are in the rough ballpark of what NREL is assuming it will cost and what they're offering in support.

And that's all I have. And I was not following the questions as they were coming in.

Moderator: Great. Thank you so much, Dean, for that presentation. There was a discussion for using tools such as Noiselab for the—primarily around sound. Do you have a sense of tools, or would someone like to ask that question more formally? And then we can go from there.

Male: The question was in terms of the post-processing of the acoustics?

Moderator: Yeah.

Male: So, I have not been following, but I do think Noiselab was trying to really develop all of that, which I guess would be a—I think what I have is—they're experts in this field and they've done a lot of work on wind turbine acoustic testing. And what I've bought from them is—it's set up to do the initial processing of the data, but I think they've gone—I think their goal was to go further with that and actually get numbers—more of the processing done so that it would give you outputs that are more ready for the report type of thing. But I haven't followed up on that at all so I don't know where they're at. I don't know if there is a product that's available.

Moderator: Great. Any other questions for Dean before we go to the breakout session? Here. There has kind of been some lively discussion we've had in and around the requirements or needs for certification, and I'm not sure if we need to get into it too deeply. I mean, I think three points I would just bring up that kind of goes beyond what's in there now. The first one is in a lot of the studies that have been done, like the future study, the acceptance of distributed wind is a key factor in option rates. And so, if the public doesn't have favorable views of distributed wind, then there is not an adoption and there is not a market. And clearly, certification is one of the clear ways to get to better consumer adoption, and not-working turbines is one of the biggest problems around that. And anybody who's worked in kind of engagement with local governments know one of the first—I mean, the first comment I get all the time is "We don't think about distributed wind because it never works. Distributed wind doesn't work. It never works." And so, certification is a way to clearly make that happen.

Two other points from a DOE perspective and NREL perspective. We feel very strongly about certification, but that's why this whole process is here. That's one of the key reasons for CIP, is we know it's a barrier. We know it's high cost. And therefore, that's why DOE is willing to invest substantial resources to support the industry in covering the cost of certification, because it is a known barrier. And so, that's out there.

The last point is it has been identified—and I'd love industry kind of feedback on this, and maybe we do this after—we have this conversation after we have the next break, but we—there is a balance between cycling innovation and the certification process. But if you work really closely with your certification agent there is nothing that says that certification has to be as hard a process as people make it out. And certainly within CIP certification can be built into certain efforts. So, if you have a certified turbine and you want to improve a component—new blades, new tower—well, tower is a little different, but new blades, new rotor, new whatever—include the cost of your certification agent in that and include recertification of your turbine in that process.

Now, if you're doing wholesale changes and you have to do a complete retest of everything, then clearly you can't do that and you would need to reapply for certification. But I think generally speaking, most of those certification bodies are very open to engaging with manufacturers about small or reasonable modifications to their turbines and fitting it within the certification framework. And those costs are applicable within a CIP context, and so there's no reason why you shouldn't be engaging with certification agencies if you have a certified turbine that you want to modify with a new component or a new something of that nature and have those costs included. That obviously requires that you're engaging with them primarily before you write the CIP proposal.

OK. With that being said, the plan now is to go into our breakout groups. So, we have four breakout groups that are kind of driven around the different topical areas, so we certainly encourage people to join us for those breakout groups. And then, from the breakout groups we're going to go into another break, kind of a lunch break for people in Mountain and the Pacific. And so, if you're not interested in a breakout session don't feel that you need to go into one. Just come back at kind of 1:30 Mountain—or, sorry, 1:15 Mountain Time, 3:15 Eastern Time, and we'll continue on with this kind of Q&A session, expanded Q&A session, and also to get feedback around—about the CIP process writ large.

We have four breakout sessions: pre-prototype development, that Robert is going to be leading; component innovation and system optimization, that I'll lead; prototype—the whole kind of testing portfolio, that Scott Dana and Jeroen van Dam will lead; and then lastly, manufacturing process innovation, that Dave Snowberg will lead.

If you go up to your—the breakout box and pull that down you can see a "Join breakout sessions," and give us all a few seconds but you should be able to see these four breakout sessions in that list of "Join breakout sessions." We also have two meeting rooms that we have set up, and that allows people to just go into a meeting room if they want to have a dialogue with somebody else. You can just pop into that, have a dialogue, and then pop out of it. And so, consider that the hallway for kind of further conversations.

So, please join us in the breakout rooms. We'll kind of wrap up all the questions that we have there, we'll go to break, and then we'll be back at 1:15 after that break. So, thank you all again. Thank you, Dean, for that presentation. And we'll have more time for Q&A in a minute.

[Silence from 3:19:54 to 4:12:29]

Moderator: Hello, everybody. Welcome back. I will give a couple minutes for people to join, but the next session that we have here is one of our kind of general Q&A sessions, but it's also a time that we'd love to kind of solicit thoughts or input from any of you on how we can improve CIP and the CIP platform to be able to support the industry more widely. We did have a nice comment in here from Michael, really kind of pulling out this doing—or the ability to do testing and get testing facilities available at NREL, but I would broaden that out at the other laboratories—to the other laboratories. And to a degree I think CIP tries to do that, maybe in a somewhat limited way, but certainly if there are kind of facilities at the national laboratories or within the national laboratory complex, feel free to kind of take advantage of them and use the CIP funding to be able to cover those costs or to request specific technical support in which we're essentially doing, Michael, here what you asked, to be able to bring the kind of—the expertise at the laboratories or within the laboratories, whether that's engineering expertise, modeling expertise, or facilities to bear on some of the problems that you're having.

That being said—and I led this off a little bit at the top of the session, we really want to support the distributed wind industry, and that includes the consultants within the distributed wind industry as well. And so, we really try to have you identify kind of expertise outside of the DOE complex first. And so, if you want to develop a fast model for your turbine or something of that nature there's lots of great modelers out there, both in industry and in academia who would be more than willing to help you go kind of through that process. And having NREL, as an example, do that work is not really appropriate because that would put us in direct competition with consultants and other people within the industry. So, we really try at the first level—let alone your engineers for that matter, engineers within your company. So, the first level is to really have it done at your company or with consultants within the distributed wind industry. But if there are expertise like blade testing or something of that nature that is really kind of specific to the laboratories or, as an example, at the NWTC but also at Idaho National Laboratories, there's great micro grid capabilities that would allow you to test wind turbines or power conversion equipment at varying voltages or under unusual circumstances, in which case don't hesitate to engage with those entities, the laboratories, to be able to provide that support. And Scott will talk about this at the end of the session today, giving you some examples of what that technical support could be.

But certainly include that in your proposal, because the facilities—Michael, as you mentioned, the facilities are here for the support of the US industry writ large, and wind industry in this specific case. And the only way it's of use is if you can come and use it, and CIP is definitely an avenue to allow that to happen where you don't have to pay for it. DOE pays for that time, that engineering work to test your products in a way that you want to have them tested.

So, hopefully I've asked that question, though, Michael, don't hesitate to jump on. Other folks, if you would like to type in a question here or if you would like to just unmute yourself and pop on, we'd love to hear from other folks. I'm sure everyone is tired of hearing from me and us NREL-ians.

Joe, you had a question here. Do you want to ask it? If I don't hear from you in a second I'll just read it out. So, a question relating to supporting novel turbine designs under CIP. "Could a turbine be less than 200 swept area submit for a type certification solicitation considering some of the additional [audio garbled] or even if they [audio garbled] as opposed to AWEA [audio garbled]?"

I am going to punt this over to [audio garbled] to answer this question. Brent? Robert? Do you want to answer that one?

Male: It was garbled, sorry. I didn't hear my own name. Give me a second.

Moderator: Brent, Robert, or Jeroen? You're better—my general sense is yes, you should be able to. But clearly read the solicitation to make sure. But if you felt the turbine needed to undergo different certification that's possible. [Audio garbled] to justify it. So, doing IEC in place of SWT-1, even though you were focused on the US market, I'm not sure why you would want to do that but we'd probably not be overly supportive of that because SWT-1 is designed to be easier and less expensive. But if you could justify the reason for not doing it I don't see a problem with that. But Brent, Jeroen, or Robert?

Male: Yeah, so we had the question in the breakout group—this is Jeroen—and I kind of quickly looked through the solicitation and there's nothing that precludes it other than the title of the topic area, which says "larger than 200 meter rotor swept area." And so, we just wanted to make sure that that by itself would not disqualify the proposal. I think technically there could be very good reasons to go that route.

Male: —myself here. You can't hear me when I'm muted.

Moderator: It's hard.

Male: Yeah. And we look at these things every year, so we may address this. You never know. Also, if you notice that—there's somewhere in there that it says that if you want to take exception to something you can do that. There's a risk to that because if it isn't well enough justified we may say, "Well, this is not responsive." But if you take an exception to something and justify it, then it will be considered. Again, it's a risk. But if it—if the potential reward for you looks better than the risk, then you're welcome to do that.

Brent, were you trying to say something?

Male: No, I don't have anything to add. I mean, technically the—only that technically the -1 and site certification has no bottom end, so —

Male: Right.

Male: But that's separate from the CIP stuff.

Moderator: Great. Hopefully, Joe, we answered that question. And I guess—because I was informed that everybody is muted and only the host can unmute attendees, if you raise your hand—and that's a little thing above the audio button, or it should be above the audio button—then Rachel can see that and unmute you and you can ask the question.

There is a question in here from Ian talking about kind of a sliding scale based on swept area. That's kind of a great idea. It's something that we have noticed, especially with SWT-1, going from very small up to 150 kilowatts it's pretty clear that a one price doesn't fit all in relation to that. So, some sort of allowance for rotor size would make sense. So, definitely when the new CIP solicitation comes out—assuming it comes out—definitely take a look at that and make sure that we're—yeah, just take a look at it.

Male: Yeah, the other thing is that, Joe, we created some different classes in SWT-1. There wasn't much distinction in the work that was required to certify a half-a-kilowatt machine and a 10-kilowatt, or even a 30- or 40-kilowatt machine. That's changed a lot. So, if you're looking to certify a half-kilowatt machine now we would not expect you to put in a proposal that goes for the maximum for certification.

Moderator: Great. Robert, since we have you on, Daryl had a question here about the kind of—the testing and certification for different size turbines. He put in here 500-, 1000—sorry, 750-, and 1000-watt turbine, and how those be addressed within certification. And Brent, you might—I mean given your experience in the Small Wind Certification Council, how a certification entity might address a family of turbines but at different powers but similar designs.

Male: I think it's a better one for Brent to do.

Moderator: OK.

Male: Yeah. There I am. We—SWCC did have some language in there about a family of turbines. You get an immediate discount when you apply with a family of turbines. But the details are grayish. If there's a way to do the duration test on the worst case scenario and not test every one, then that's a savings. But mostly it was just reviewing a package of similar turbines was more efficient than a group of very different turbines. So, when we look at the details, different rotor sizes usually means each size needed its own testing. But we—it is more efficient to sort of certify and analyze a family, but you can't just test one and scale up and down.

Moderator: And then, from a proposal process, though—I mean, we can certainly take this under advisement—I mean, based on that, the kind of you test each—at least on some level, right now CIP certification testing is for a specific turbine. We haven't gotten into a question of how much variability there might be around that. But right now CIP is designed around one test system. Again, you're talking about very small systems, Daryl, and so the cost delta might not be too high to test a number of units. But you still have to go through that—kind of the pretty rigorous testing on each one of those units. But it certainly would be a conversation to have with the certification agent, _____ certification agent or one of the consultants, like Joe as an example, who has a lot of time doing this and could provide some guidance.

And then, I think I can address the second question here: "Do we need different proposals for each?" Under—historically, the answer is yes. So, if you were testing—certifying two different turbines—we actually had this with Primus. They had two units—they were slightly different, but they had two units that they wanted to certify. And after talking with the certification agent they had to go through a test protocol for each unit. And they submitted two applications—in different years, but submitted two applications and tested both of those. And both have been verified.

In truth—and Dean or Brent, you can dive into this—the cost of the turbine is not necessarily the highest cost element of the certification testing process. It's the data crunching and the sensors and all of that kind of stuff that really drives it. So, unfortunately there's not a lot of cost savings.

Male: In fact, for the Primus turbines, because of the small size it was difficult to get some kinds of data and to set up sensors. And so, it in some ways cost more.

Male: But yeah, it doesn't correlate very well. We did—actually, we did testing for the Air Breeze, and Dave—EOCC—witnessed the testing, and the same engineer that just left the largest turbine in the world came to _____ Mountain to look at the smallest turbine in the world, and that guy costs the same. So, it's not very correlated to rotor swept area or turbine size.

Male: Which is one of the reasons we modified SWT-1, so that we diminished the amount of testing required for the smallest turbines.

Moderator: OK. I don't see any other hands raised, but please raise your hand. or type in a question. There is a question here from Joe in regards to the—kind of the type of proposals or the numbers of proposals that we see. Generally speaking—and obviously it varies year by year—we see a lot of the pre-prototype proposals, as one would expect. Kind of new ideas. A fair amount of component innovation and optimization. Not very many manufacturing and not tons of the kind of the certification testing or type certification. And then, the prototype testing ebbs and flows. So, some years we get some and some years we don't get many.

All that being said, all of the proposals are stacked against each other. And so, you're not—you are graded differently, whether you're a prototype or a certification testing. There's different grading criteria. But the final scores are basically equal across all of the different topical areas. And so, just because we get more proposals in one area than in another we don't necessarily—we don't' clearly fund one in every area. And we pick the best proposals overall. There is a little bit of wiggle room in regards to trying to make a breadth of different proposals. And then, as everybody knows, they have different amounts and we have a fixed amount of funding that we have. And so, there is a little playing around with the—which ones we can fund based on diversity of manufacturers and the money that we have. Generally speaking, you're ranked by your total score, and if you get a really great score it doesn't matter what category you're in.

Does that answer your question? You can type it in—or, Rachel, if you could unmute Joe—if we wanted to talk a little bit more about that.

Male: No, that's it, Ian. I just—I heard a couple people asking questions trying to figure out where they fit, and I thought if that was something you could share that was possibly helpful for people trying to plan which solicitation they want to go to, or maybe if they feel they're kind of in between one or the other, maybe it can just kind of help them best prepare for what's upcoming.

Moderator: Yep. Thanks, Joe. I mean, it is quite competitive. I don't remember, how many proposals did we have this last year? We had about—I'm not sure of the exact number but we had about 35 proposal and we funded 8 of them. So, not a high kind of a percentage. But the score—if you drop off all of the proposals that were pretty low quality—so, I mean, we got one that was a PowerPoint presentation, so that doesn't necessarily count too much—if you take the really kind of high quality proposals there were a few proposals that we would have loved to have funded but couldn't because of the monetary resources that we have devoted to CIP. If you write a really good proposal the chances of you being funded are quite good.

Male: Cool. Thanks, Ian.

Moderator: Also, would love kind of additional comments or anything of that nature. And again, raise your hand and Rachel can mute you. But I said at the top and Brent said it at the beginning: CIP is for you and we administer it, but our goal is to provide it based on industry comments. And I think Mike Bergey is on—was on, and I'm not sure if he has a comment that he would like to make around this, but we're very receptive to industry feedback. And if there are things that you would like to do that you don't see how it fits into CIP or you're worried about shoehorning it into CIP, I can't promise that it will be incorporated but let us know and we're more than happy to try and make that happen.

Male: Yeah, the pre-prototype category, topic area, was added just because of that kind of request two, three years ago.

Male: Ian, this is Mike Berdan. Listening to your comments about the quality of the proposal, a little question: Do you guys not look at the content of what the proposal is about, which should have a higher, I would think, meaningful than someone who can write a proposal—it's—there are some good people out there with great ideas but the proposal writing is obviously very specific. We don't know what you like. What do we cater to? I've written proposals for military contracts, but if I sent that same proposal to you, you'd probably look at it and go "What the heck are they talking about?"

So, I would think you should put more emphasis on the content of the proposal than the proposals themselves. Otherwise, I have to go out and find someone who is really good at writing. It gets kind of like a resume. People who write your resume make it look fantastic but the content is really not very much. So, would you prefer to have the person tell you the truth about the content? So, I'm a bit confused when you say how important the proposal is. I wish it was the other way around. I wish the writing of the proposal was not as critical as what the content is proposing. What would you say about that?

Moderator: Yeah, no, no, definitely. And so, thanks for asking that question because I want to be really, really clear about this. So, when I say a good proposal I say—that to me means a proposal where someone has reviewed the criteria in which the proposal is scored and answers all of the questions reasonably well. So, it doesn't need to look pretty. It doesn't need to have tons of fancy diagrams or anything of that nature. If you address every scoring criteria that's in the RFP, even if you write it out by hand, you will get a good score. Or generally.

Now, the people who review that are a bunch of geeky engineers, and so we don't write well either, so we're not going to grade you for bad writing. But you can't try to hoodwink us because it's not going to happen—I mean, maybe it will happen, but generally it won't happen. But we spent a lot of time really trying to get the criteria to invite you to provide the type of information that will end up with a good proposal. So, you'll document it. You have a path to certification, or at least you're thinking about it. The criteria for LCOE for the cost of energy calculations, there's a spreadsheet there that allows you to calculate it, and you can put it in detail in regards to where you get your costs from and what your power curve is and how you calculate it and all of that kind of stuff.

And if you articulate that and can demonstrate a reasonable cost of energy improvement from your technology—again, it depends on which topical area you're proposing under, but if you articulate that and answer all the questions, then you'll get a pretty good score. The people who don't get good scores are not necessarily people who write good proposals—or they write poor proposals and just leave sections off. So, they don't answer the question about certification because they haven't thought about it. So, they don't mention certification in their proposal and they get zero points for that and they get a very poor overall score.

So, certainly having a good proposal writer doesn't hurt, but if you answer all the questions truthfully—I mean, not like people would lie, but with kind of backups that geeky engineers can understand, then you're going to end up submitting a good proposal, even if you wrote it out [audio dropout].

Male: OK. Thank you.

Male: This is Mike Bergey. I'm going to rise to the bait there and just point out that the CIP program was proposed by the Distributed Wind Energy Association and accepted and it's been running now eight or nine years and is being shown around the Department of Energy as a model of the kind of program that targets taxpayer money in a very effective way to create competitiveness and jobs here in the US. And if you want to have a voice in the kinds of policies and programs that affect businesses in our industry, you should take a hard look at joining the Distribute Wind Energy Association. And you can get information on that at DistributedWind.org. We're a very strong proponent of the CIP program and we work hard with Congress to fund this and similar programs, and we've been successful well beyond the size of our industry. It's a good organization and we appreciate the opportunity to work closely with the management and staff at NREL and the Department of Energy, who are really a bunch of whiz kids who are working hard to help us have a better business environment. So, we appreciate that very much. But do look into joining DWEA if you're not already a member.

Moderator: Great. Thanks, Mike. Daryl, you had this question: "Is it industry professionals reviewing these or is it NREL/DOE/NSF who's doing the reviews?" The answer is it's NREL doing the reviews. Part of that is contractual. These are NREL contracts and therefore NREL employee have to do the reviews of them, and that's a contracting requirement. We do as needed bring in kind of other people to provide—to solicit kind of information if there is an area that kind of goes outside of our expertise. But—and so we do that if we need to. But it's NREL people that are doing the reviews. And it's NREL people like the folks that you have here on the phone who have years and years of working in the industry, some of them in different roles—building turbines, starting companies, those kind of things. So, the reviews are more technically based but clearly market is—and path to market is an important element of the process.

A question from Padma. "Could we get a cheat sheet for a cost—for cost proposal?" Why don't we bring that up with Kyndall when Kyndall is on the line—or, Kyndall, if you're on… We probably can do that. I'll have to talk to Kyndall about that but my sense is we can.

Kyndall: We do provide the cost proposal form. When you are looking at the RFP there's a link to all the forms and there's a template that you can use, if that's what you mean by a cheat sheet.

Moderator: Or, Padma, are you thinking about some kind of help in filling that out or—not help; that—I don't want to be derogatory—something that helps explain what is expected in the different boxes?

OK. So, we'll look into if we can help create a cheat sheet or something to make that governmental form a little less painful.

Male: A guidance sheet.

Moderator: A guidance sheet. Yeah. Great. So, Daryl, I see you have a comment here. It probably would make sense to take that kind of conversation offline. Again, I really want to say—well, all of us have, as you put it here, moats that we need to—that are out there. And God knows I'm trying to contract with NREL; I pity the people that have to do that. No offense, Kyndall. You work amazingly hard at it, but it's not a simple process. But we're here to try and help the industry. So, Daryl, if there are things that you see that we could try to address—I mean, we can't get rid of the moat per se but we might be able to make a bridge. And so, we want to do what we can to help make those bridges.

Any other questions or comments? Going once… going twice…

All right. So, let's without further ado—oh, I didn't even notice, Kyndall, that you were next on tap. So, Kyndall is going to provide a quick overview of the contracting requirements and things of that nature to give us some heads-up around that, and then we can have come questions, Q&A around kind of the contracting process and hopefully address any of the mystery there.

Kyndall: Sounds good. Hopefully, everyone can hear me.

Moderator: We can see you and hear you. Yep.

Kyndall: OK. And are my slides up? Because I don't see them. But perhaps you do?

Moderator: No. And it's —

Kyndall: I can—don't see them either.

Moderator: Oh, no—oh, section six: "Design Evaluation."

Female: I thought section six was Kyndall's. Kyndall, do you mind uploading those again?

Kyndall: Sure. You can just —

Female: OK. I'm sorry about—sorry about that confusion.

Kyndall: No problem. Where do I go to upload them?

Moderator: Share—yep. Share.

Female: Top left. Share.

Moderator: File. And now we'll select it. It's probably easier if it's on your screen or something of that nature or it's on your root drive to find it. And then it should only take a second to upload it.

Kyndall: OK.

Moderator: While Kyndall does that, any additional questions?

Kyndall: OK. That's better. OK. I see them now. Do you see them?

Moderator: Yep.

Kyndall: OK. Great. OK.

Hi, everyone. As Ian said, my name's Kyndall Jackson. I'm a subcontract administrator with NREL's acquisition services department and I've enjoyed working on the CIP projects the last few rounds. Today I'm just going to be going over a little bit of the proposal submissions timeline and—as well as some of the other requirements.

So, if and when a request for proposal is released it will be posted on the beta.sam.gov website. That used to be FedBizOpps, if you're with that. It did switch over to beta.sam.gov about a year ago. If you are on the interested list of vendors you will get notified when that gets posted. So, if you're not already on that list please let me know that you'd like to be included.

The RFP document itself is a pretty lengthy document, about 15 pages, so it has all the details for how to submit a proposal, including the contact information, who it needs to go to—likely myself—the requirements, due dates, et cetera. That should all be in there.

As far as past rounds, the timeline, typically the RFP has been released in late February with a submission due date in March. I know we had some extensions this past year due to circumstances with Covid but hopefully this next round will hopefully be more in line with what we've seen previously. And then, the awardees are usually notified in the summer, with the goal to complete negotiations by the fall.

There is an opportunity to submit technical questions. And again, the date for submitting those will be specified in the RFP. Once we get all of the questions NREL will provide responses to the questions and an amendment will be posted to beta.sam.gov—so, to the same website that the RFP is on. Once we receive all the proposals they are evaluated against the best value selection process, which takes into account both qualitative merit and price. Within the RFP we'll see the merit criteria—so, you'll see per each topic area the evaluation criteria that it's going to be evaluated against as well as the percentage that each criteria section is weighted, is evaluated how much weight each is given.

As far as the process, there's an initial evaluation just to make sure that we've received an acceptable proposal. So, at that point you may be asked—offers may be asked for clarifications at that point. Once we've narrowed down all the proposals that are acceptable, they are then evaluated against a statement of work and the merit criteria, at which point offers may be asked to engage in discussions with NREL. And then, once all the proposals have been evaluated and scored, successful and unsuccessful offers will be notified, at which point we will enter into negotiations with the successful offers in order to make an award.

Subcontracts have been put in place as firm fixed price with price participation. So, an example from past rounds, it will be identified in the RFP how much price participation is required, and with previous rounds it's been 20 percent for most topic areas. But again, that would be clearly outlined in the RFP. The price participation, one of the requirements that you'll be required to submit is a payment schedule, and this is where you can demonstrate that price participation that you're proposing. I'll explain that a little bit more in the next slide because I have a little table.

And then, the common type of price participation can include labor, whether a direct labor or lower tier, equipment, and supplies. And then, just a quick note on equipment: If NREL—if within your proposal you're proposing the purchasing equipment and if NREL is paying for the equipment, since we're using taxpayer dollars we would assume ownership of that equipment once this effort is completed. So, if the offer wishes to retain ownership of that equipment after the effort is completed you would want to include that in your price participation.

As far as allowable costs, they need to be considered reasonable and allocable under the terms of the Federal Acquisition Regulations and DOE regulations, and I've included that FAR Part clause, 31.201-2, which you can just Google and that will give you more information on what is considered reasonable and allocable and allowable under the regulations.

There is an opportunity when you're submitting a proposal to specify any proprietary or restricted data. So, this can be in the form of—as a disclosure statement on your proposal, as well as within one of our forms called the Representations and Certifications. That's where you specify that information. It is just important to make sure that if you are stating that any of the data is proprietary or restricted that it is in fact that. If we get something where it says the whole proposal is considered proprietary or restricted, that will usually be kicked back by Legal. Anything that is common knowledge within the industry or available via a website or something, we would not consider that to be proprietary or restricted. So, please just be sure that it is in fact one of those categories if you're listing it as such.

Part of the proposal, we also ask that you provide your acceptance of the statement of work as well as NREL's terms and conditions or provide requested exceptions.

And then, the remaining forms that are required for submission are the price cost proposal form, which I was just mentioning. It is a template but—I believe it's a five-page document. The first two pages—Padma, I believe you asked that question—are the actual template. And then, hopefully in pages three, four, and five there is kind of a breakdown, a description of each category that hopefully explains what it is we're looking for. But again, if there's something specific that you'd like further guidance on, just let me know and we can perhaps include a note in the RFP document to further explain that.

And then, an organizational conflict of interest form. So, there's two of these; you should only be submitting one. A representation would basically be a statement saying there are no organizational conflicts of interest. A disclosure would be if you thought there could be a potential conflict of interest, in which case you would fill that out.

If you're a new vendor to NREL, you would need to submit a W-9 and an ACH banking form. And then, the last form is the representations and certifications. And the one thing I'd just like to point out on this is you must be—all NREL subcontractors have to be registered within the system for award management: sam.gov. And I'd just like to point that out because it can take a couple of weeks to complete, so best to get started on that sooner than later.

So, here is that sample payment schedule I mentioned before. So, because this is firm fixed price, subcontractors are paid based on the completion of the deliverables. So, in a proposal you will want to show in your payment schedule—you'll basically assign a value to each deliverable based on the amount of effort you think it will take to produce that deliverable. So, per each deliverable you'll have a value assigned to it and you'll want to show both the NREL portion and the subcontractor portion. And that's, again, where you can demonstrate the price participation where you're meeting that minimum that's required.

And then, as far as post-award with invoicing, again, payments will be made based on pre-negotiated values in the payment schedule upon submission of approved deliverables, and they will be submitted to Accounts Payable. All the instructions are provided in the subcontract award.

So, that is all I had for my presentation. But are there any questions for me?

Moderator: Any questions for Kyndall? Thank you, Kyndall. I don't see any here.

Kyndall: OK.

Moderator: Any raised hands or…?

Kyndall: I do not see any.

Moderator: Yep. One last thing I would put out there is, like the rest of us, Kyndall is definitely part of the team here. And so, if there are any questions that kind of come up don't hesitate to reach out to Kyndall from a contracting standpoint.

Kyndall, you are the only one who is allowed to talk to people about proposals once an RFP goes out. Once the RFP is out, if people have questions about how to fill out the proposal or the forms or something of that nature, is that something you can help them with? Or are you not allowed to engage at all?

Kyndall: Yeah, absolutely. If you do have questions about how to fill out anything or what to submit you can certainly send those to me. And if it's something that we think will help a wide range of people, then we may—I might include that in the amendment as well, answers for technical questions, just in case we think it might be a point that multiple people are confused about.

Moderator: Great. Thank you.

Kyndall: Yeah, thank you.

Moderator: No questions for Kyndall?

Kyndall: It looks like maybe one. Presentation… yes. Nate, I will—yes, I will add you to the list. Anyone else that wants to be added, if you'll just send me over an e-mail to—hopefully you can see it—kyndall.jackson@nrel.gov—I will get you added.

Moderator: We promise not to spam you with stuff. We only send out stuff really targeted at CIP. So, stuff like the webinar, the notice of intent, and then the RFP itself. And then, if we do anything like modify or extend the proposal deadline like we did last year due to [audio dropout]

Kyndall: Sounds good.

Moderator: Great. Thank you very much, Kyndall.

Kyndall: All right. Thank you.

Moderator: Great. Next we'll—we're now ahead of schedule. Thank you, Kyndall, for catching us up.

[Laughter]

Kyndall: You're welcome.

Moderator: We'll move on to session seven, the design evaluation, and Robert.

Male: Yeah, so first of all, I apologize. I didn't catch up when we shuffled the presentation order, so this is marked session six and everybody is confused. But—and I acknowledge Rick Damiani, who prepared a lot of these slides several years ago, and I just continue to use many of them with a few of my own and a few modifications over the years.

Let's see. Oh, no. Ah. There we go.

Let's see. All contracts for certification have a go/no-go design review. Now, this is not the same as done for the—by the certifying body. This is sort of a pre-evaluation and it's primarily used to make sure that the design is likely to be certified if it passes the field test successfully. And it's based on experience. We've seen designs that they had a weak link and they were—they were not failing in the field easily but they didn't have the safety margin required in order to beef that part up. Everything had to be redesigned in a whole subsystem because there was no room physically to make it bigger and it would have been a waste to start field testing that turbine. So, this is what we're trying to avoid. Also, sometimes we find that folks really haven't done sufficient engineering to be able to pass. That's why we ask for loads at this point.

The design review done by the certification body will be done after field testing because they use the input from the real testing to modify some of the loads results—or confirm them at least. So, they don't do it ahead of time. We have to work with what data you have available.

For small wind, less than 200 meters squared or less than 30 kilowatts for SWT-1, assuming that the draft version gets accepted, which seems highly likely at this point because it's passed the big hurdles, you have three options for certification. And—well, OK. The simplified loads equation is the option that's only available for under 30 kilowatts in SWT-1 and recommended for 10 kilowatts and less. It has very high safety margins because essentially it's designed to produce a robust enough turbine that even if the turbine is really poorly behaved it's still robust enough to survive.

Aeroelastic modeling, which needs at least some validation, has lower margins because you're actually looking at the loads on the specific design that you're working with. And then, VAWTs—there aren't a lot of aeroelastic models available for VAWTs, so they tend to mostly end up doing load tests. And that is difficult, expensive, and very time-consuming because you literally need to take enough loads measurements at each necessary condition in order to statistically look for what would be the worst case.

Simplified loads are limited, approach is limited to HAWTs, rigid hub—teetered hubs for instance. If you had a teetered hub, two-bladed machine, that would not be eligible. Cantilevered blades. You can't have guy-wires to the blades. And collective blade pitch. Not that you have to have blade pitch. Certainly stall control machines are more common. Furling. Any number of other approaches to power control. But if you have individual blade pitch control that's not allowed in the simplified loads production.

It doesn't cover all the loads explicitly for each component. And so, you have to work your way through the machine and say, "Well, is this load that I calculated for the shaft, main shaft ending, does it transfer into this component?" And I have to look at that component.

It doesn't do a very good job of capturing fatigue. So, people often say, "Well, the yaw rate is so high that the yaw moments, the yaw-induced end of the blades and shaft are ridiculously high for a nice, well-behaved turbine." And that's true. But if the fatigue loads are underrepresented and unbalanced usually they kind of cover each other. Not a great system, but it's simple and easy to use. And personally, I think it's a great first cut at coming up with loads to start to do the preliminary design, unless you have a quite small turbine where the amount of materials that it causes you to add over what really is needed isn't significant. Again, it's being used less and less. It does require some data from testing—RPM, for instance.

Inputs. Again, rotational speed at the design point. That has to come from field data. You can estimate it if you don't have it yet, but really, by the time you're going in for certification you should have been testing a prototype.

Power—again, from test. And while your measurement of power may not be as good as the official test site, you should be good enough that the official results won't change your calculations a whole bunch.

Let's see…

Aeroelastic modeling. This is really the heart of what we expect to see in most cases. It's individual to your turbine, so if you design a well-behaved turbine that doesn't have extreme motions under some circumstances, it allows you to take advantage of that and design your structure appropriate to the loads that your configuration of turbine will experience. And because it's much more accurate the safety factors are smaller than would be for the simplified loads. And there's a lot more fatigue calculations done, so you get—you don't have to beef one thing up to cover the fact you aren't doing a good—a clear job of getting good information on another area.

Let's see…

Full-scale load measurements. Again, unless you absolutely have to and this is the only way, you don't want to do this. You'll probably want to do some measurements. It's really great to have some measurements to validate your aeroelastic model. On the larger small wind turbines, once you get over about 30 or 50 kilowatts or so, you're probably going to want to do blade bending. That—if it isn't coming from the blades it isn't getting into the turbine for the most part. So, if you validate your model with blade loads, then you've got a very high level of confidence in your model.

Now, I'm not going to read through all this. There's three slides here and the reason, the only reason I kept them in is because they're really great information, and since this will be posted online you can read through it when you're looking at preparing your analysis for getting your turbine certified and really get to understand what it is that you've got to put together. And it's pretty damned expensive—extensive. [Laughs] Slip of the tongue there. So, there's three pages of this detailed list of what you're going to need to provide.

Control system is part of the review, so you really need to document your control system and understand and be able to answer all the questions about, OK, well, if this fails does the turbine go to destruction or do we have a safe—do we go to a safe mode? And how does that happen? While I don't think you're required to do failure modes and effects, you really need to do that to be able to answer all the questions. Yeah.

Each of these—the simplified loads method has a set of design load cases. And the aeroelastic model has—also has, but they're different because there are things that you can do with the aeroelastic model that you can't do with the simplified loads model.

One of the—you also have to look at the loads on any service condition or—and during transportation. So, for instance, if a service condition involves tilting the turbine down you need to look at the loads involved in tilting the turbine down. If it involves picking up the gearbox out of the turbine with a crane you need to look at those loads, both for the gearbox and now you've got a turbine that may be very unbalanced as far as having the weight over the yaw bearing. Is that yaw bearing designed to take that load imbalance if you take the gearbox out? And then, transportation. Unloading it from a truck. Moving it around with a forklift. Is everything strong enough for that? Is it all built to be able to be moved and done so safely?

You do have to have a manual for the turbine for installation and service and maintenance. That's part of the package that needs to be provided.

OK. Here's the design load cases for the simplified loads method.

And then, here's the more complex maximum loads for different components and different orientations.

Now, I would not go about putting all of this analysis together myself if I was designing a turbine. I've never been a fast developer or designer. I haven't done a lot of that kind of modeling. And the people who do it are—who do it a lot are just amazingly good. They've got—they've built programs to help them run multiple load cases, one after another. Just—the computer will run one load case and then it will increment to the next variable that changes to run another load case and it will run that. So, they can be much more efficient at doing this kind of work. If you have enough money and you want to develop that capability in-house, that's great too. But it's a big—it's a fairly good-sized lift.

This is a Campbell diagram. Essentially, what you're looking for here is are there frequency response resonances in the structure with the 1P, which is every rotation of the machine; 3P, which assuming you have a three-bladed machine would be every time the blade passes, every time a blade passes. For instance, even if you're in an upwind machine there's an impact from passing in front of the tower because the wind is slower in front of the tower. So, there's going to be an input every time a blade passes in front of a tower. Is that going to excite your tower and make your tower start to get—start to move around way too much? So, you're going to have to look at that.

And the—there is a static blade load test required in both the IEC 61400-2 and SWT-1, unless you're under one kilowatt for the SWT-1.

Now, there's a lot of documentation to a design review. And we expect to go through that and perhaps have questions and ask for more information. That's not a sign there's a problem; it's just we want to confirm or make clear something that is not clear to us from what you've provided. So, don't get concerned if and when that happens, which it likely will.

One of the things that we've done in the RFP is we require folks to check off that they have the appropriate standards for which they're—to which they're working to, because in the past we've found folks that just really hadn't read the standards but they were applying for certification testing, and they weren't at all prepared. So, please, if you can't afford the standards you probably can't afford to get something certified, so get the standards. You need to understand what you're dealing with, otherwise you haven't got a chance of success.

The alternative is to hire a consultant to do the standards work with you. I assure you, buying the standards is a lot cheaper. And you may find the consultant saves you a lot of money if they help you focus in on getting everything ready and they know how the certification bodies work, what they expect. And if you send in incomplete stuff to the certification body and then they have to come back to you and come back to you and come back to you, that will—they will charge you for that. So, you may save money by hiring somebody that can go through it with you beforehand that knows exactly what a certification body is looking for.

Let's see. I guess that's it for me. So, I have time for some questions.

Moderator: Thank you, Robert. Were there—Daryl had a question about finite element analysis. Daryl, do you want to—can we unmute Daryl and he can ask the question? Or…

There you go, Daryl. I think you can speak now. You should be able to.

Male: He may have muted himself too.

Moderator: Yeah, we can't hear you, Daryl, if… Not quite sure what that refers to but, Robert, if you have a sense…

Male: I said I am good. So…

Moderator: Oh, OK. Great. OK. Great.

Male: OK.

Moderator: Any other questions for Robert? After being behind, now we're well ahead of schedule.

Male: Yep.

Moderator: Anybody else? Any other questions? All right. You don't have to hang on here.

So, let's move to session eight for some additional considerations for CIP technical support from Scott, and back to Robert for a couple of other points, verification questions.

If Scott is here… I think your thing is up there.

Male: OK. Can you hear me OK?

Moderator: Yep, we can hear you.

Male: OK. Excellent. Yeah… yep. Here it is. Hi, everyone. I'm Scott Dana, NREL Field Validation Engineer. I've been involved with CIP for a number of years now. And we'll go over some of the technical support areas that NREL can be helpful with. This isn't a solicitation of our help. I think Ian had kind of touched on this before, that we really don't want to be in competition with industry, so we do encourage you to find those industry partners that have capabilities to meet your needs, whether it's with design codes or dynamometer or structural testing, regional test centers for certification type testing, accredited testing. So, we do encourage that. But there are those times where those expertise are not readily found or available, and NREL is certainly here to be of assistance when it make sense. So, that's what we will review here today.

I recognize many of the names on today's call, on today's meeting, so I know some of you have been to the site. For those of you that haven't, the National Wind Technology exists at the Flatirons campus, which is kind of a newer designation of our site that's in place for the last couple of years. So you will probably hear "Flatirons campus" more readily used than NWTC than in the past. We are a campus of the National Renewable Energy Laboratory. We do have numerous sites on the Flatirons campus. It's a 305-acre site, which not all 305 acres is available but we do have dedicated areas for distributed wind technology, small wind turbine sites, midsize turbine sites. Most of these are really well-prepared for easy deployment with MET towers. We do have data buildings—data sheds, as we call them—where we can house electronics and data acquisition and it's all quite customizable to fit your need.

At the Flatirons campus we do experience—we kind of have some unique wind resources—we do experience some extremes, often in excess of 90 or 100 miles per hour every season. And it's pretty unidirectional, similar to Dean Davis' site, Spanish Fork. In the lower right photo you can see what's called El Dorado Canyon. That's typically where our high winds are sourced from. And we have a wind season of about six months, so it makes it a good choice for having down time, calm time to deploy and get experiments set up, tests set up, and then execute during our wind season.

In addition to the field sites we have numerous structural facilities. In the photos here you will see a tower, a small wind tower undergoing some structural testing, some fatigue testing. In addition, there's a blade in the lower right undergoing some static pulls for ultimate loads. These, again, are fully equipped. We can do loads and force. We can measure strain, displacements. We have nondestructive evaluation systems and methods, like modal testing acoustic emission, digital image correlation, so we can get fairly fancy if the need is there. Again, just general examples is a blade and tower static and fatigue. That's pretty common, kind of our bread and butter, what we've made quite a living on at NREL.

We have dynamometer capabilities. It specifically targets the smaller turbines within the distribute wind space. It's a 225-kilowatt facility. They're a 300-horsepower variable-speed motor. So, this was ideal for gearboxes, generators, even an entire drivetrain.

In addition, we can configure the system or the, I guess in this case the drivetrain or generator and power electronics to be incorporated with the controllable grid interface. I'll touch a little bit more on that, but in that way we can actually test very specific grid events—ride—low voltage ride-throughs, changes in frequency, power, things like that. So, that's a bit of a unique capability that has come up in the past.

Yeah, again, grid testing facilities. So microgrid testing. We have these capabilities on site. The CGI, as I mentioned. We do have a new CGI that is dedicated to distributed wind. It's coming maybe—I don't know, maybe, Robert, you can mention a little more on the timing. I'm not exactly sure when all of that's going to happen, but it will be connected to one of the small wind testing rows at Flatirons campus. And again, we can literally create any type of grid event. There's grid compliance testing. And we can certainly handle hybrid systems, so we do work with—in the solar space quite a bit as well. And we can test those, your turbine and your PV system, at the same time.

Yeah. So, that's a little—that was a little bit of an overview. The physical capabilities in terms of actual technical support for design and controller space, we do design reviews often. They're often part of the CIP process, the award process. And we're certainly available for additional support in design reviews. We can do very unique component testing. We've had some things come through CIP to test a hub in particular, so it's not just the big things like blades and generators. Even the smaller parts of the turbine that need advancement can be tested at NREL.

We are creators and gurus of OpenFAST. It was formerly called FAST; now we are at—the latest version is known as OpenFAST. This is an aeroelastic design tool developed by NREL. We are experts with—in the standards, as you've heard some today. And many of those involved in today's call are actually developing those standards, so please reach out if you're struggling. It's not always easy.

And then, on sort of the softer side with cost modeling and analysis we have dedicated groups that do that every day. We are experts in instrumentation, instrumentation selection, configuration, getting creative with design of experiments. We do that routinely. Test site requirements—again, going back to standards. We have a world-renowned controls team with many controls experts. We have also developed an open source control design tool called ROSCO. That is something that I know has been made use of within CIP as well, so please reach out if you're needing help on the control side. And again, structural analysis, component analysis. And then, your Golden campus housing the Energy Systems Integration Facility can do very specific electrical type of testing and component testing.

Yeah, that's it. Are there questions now? Or should we roll with Robert's sections? I guess I'm asking Robert.

Moderator: Yeah, why don't we pull up Robert's presentation. But while that happens, if there are any questions?

There was a question from Rick [inaudible] induction generators and ROSCO. Do you know if that has been updated, Scot, or who Rick could ask that question?

Male: That's a really good question. I don't know. I mean, Nicar would be probably the person to ask. I certainly don't mind asking on Rick's behalf.

Moderator: OK.

Male: OK.

Moderator: Any other questions?

Male: Can you hear me?

Moderator: Yep. One thing that I would add that you didn't mention was cybersecurity and what's going on. That's certainly a new area that's kind of gaining need and more requirements. And that's certainly something that the laboratory complex—NREL has expertise in that area but other labs have expertise in that area as well. But that's another thing I'd add to the list.

Male: OK. Excellent. One part of a CIP contract is that there's a deliverable or current schedule. And these are the things that you'll actually complete to get paid for. This is just sort of—to give you an idea of what it looks like. Typically—oh, what happened?

Moderator: Did you just [inaudible] instead of—oh, no, you're at C.

Male: Let's see. What happened to D? It just disappeared. Huh.

Female: Robert, I think that is a WebEx glitch. I'm really sorry. Would it be possible for you to re-upload your slides?

Male: I can.

Female: All right. It does happen once in a blue moon or once during a really long presentation day. Sorry about that.

Moderator: We do have a question here about what kind of generator machine or power electronics converters are tested. So, I think that's a hard question to answer and it kind of depends on the type of testing that you want to do. And Dana, feel free to jump in here as I fill in. We do a lot of testing to specification standards and things of that nature for power electronics, a lot of it done on PV inverters. And now, certainly there are entities that do that as well—the organizations that are doing the certification testing and things of that nature. But if you wanted to do unusual things, then we certainly have that capability.

From generators it's pretty wide open. I mean, we saw the picture of the dyno, and so any types of testing, assuming that it was going to go on a dyno, we would have that capability to do. Clearly, if your turbine is a smaller turbine, then that would be something you would do in-house. The thing that's a little unique is the size of this dyno. It kind of fits in that space between the really large ones and ones that you would do in your own shop.

It might be worth having a separate conversation to get a little bit more into the nuts and bolts in regards to what you would like to be testing, what kind of research you would like to do. But that's the 60,000-foot question—or, answer. Dana, did you have other stuff to add—or, Scott, I'm sorry—did you have other stuff to add?

Male: Yeah, I mean, I think in this context it's your—it's going to be your design that's being tested. That's sort of where the technical support comes in. It would be your machine, your generator, or your electronic design, or your drivetrain configuration, even if it's using sort of off-the-shelf components. But you put it together in a way that needs to be tested on a dynamometer. That's sort of what we're getting at with that capability.

Male: Yeah, I mean, an example might be—well, first of all, you can do low-speed input because we do have—they didn't show it in the picture but we do have a gearbox. It's a large truck gearbox so it's got multiple capable ratios. So, we can gear down and drive a direct-drive alternator, for instance, or drive the input to a gearbox. And if you're trying to use an induction generator to do things that it wasn't designed for, like ride-through or things like that, that would be something we could test for, which isn't commonly available.

OK, I think I'm back on—into the show again. You'll notice that the timeline is based on the start of contract execution because that can vary quite a bit. We don't predict when that will happen. Some of it is how many questions that you raise if you're awarded a contract and some of it is just working out the details.

So, let's see…

Also, one of the things that we've added in the last couple of years is that we make it very clear that you can add deliverables. You can make up deliverables to spread the cash flow. Some of these topic areas have several deliverables early on in the first three months or so and then several deliverables close to the end, and in between it's only quarterly reports. And we don't really like to have the price for quarterly reports be huge because essentially all you're doing is telling us what you did or didn't do and what you're planning on doing in the next quarter. So, if you sent in a quarterly report that said, "Well, I was out sick this month and so we didn't get anything done this quarter," we're going to pay you for that report, so we don't want to make it a huge deliverable.

So, in some cases it works much better for you and for us if you say, "Well, given what I'm doing I'm going to finish my first prototype gearbox pattern, and that's a major milestone for me, and so therefore I'll put that in as a deliverable." So, there is that flexibility to help you spread and to actually identify the things that are serious milestones for you.

Oops. There we go.

OK. One of the things that DOE and NREL do is to let the world know that we're doing this work and supporting this work in industry. And that—it works. It's been effective and valuable. So, there are press releases and from time to time we create fact sheets about the programs and success stories. And every year we have to do accomplishments to DOE to let them know what we've accomplished. Excuse me, my throat's getting sore. And we make an effort to make sure we clear with you anything, but there are some things that we ask for as part of the process that—where you give us only things that are not proprietary, secret information. For instance, there's a one-page or couple-of-page fact sheet that we ask you to prepare right off the bat about what you're doing, and that's something we can draw on for any kind of public release.

Yeah, here we go. Here's more details. So, it says "A summary of the project" and it—[clears throat] excuse me—and it's all supposed to be stuff that's non-proprietary. So you've given us stuff that is totally OK for us to use, pick and choose here and there what little pieces of it we might need.

And one thing we really appreciate a lot is getting some images, and images that we're free to post in our gallery, our image gallery—therefore we may use them later. It's also open to the public. There is credit attached to them on who's the person who created the picture. And so, please provide us with images. They're really appreciated.

Boy, is that all? OK. That's all on that. We want to jump to the next one and then do questions?

Moderator: I don't see any questions. Do people have some? I'm not seeing hands or questions. So, yeah, I'd say go —

Male: Well, let's do my favorite topic: levelized cost of energy. So, what goes into this is basically the cost of money times the initial cost plus the annual operating expenses divided by the energy produced. Pretty simple and straightforward. There are—I'm not going to go into the details here because they'll come up in the next slides. So, we broke out the turbines into pretty major assemblies. You are more than welcome to add to these tables, make it 12 or 14 items instead of the 6 or whatever we've got here. So, don't feel constrained. We're just providing you an easy format. And this is part of the RFP, so it will refer to this. It comes as a spreadsheet. It is—the white squares are calculations and the spreadsheet is locked except for the green ones, green spots. However, we provide information on the front of how to unlock it. So, it's not that we're trying to keep you from changing things; we're trying to make it easier for you to not accidentally change things.

And depending on what you're doing, if you're doing a certification testing, then you'd fill in the baseline and you won't do anything else, because you're not improving the turbine, so we don't expect to see an improvement in the cost of energy. If you're doing a topic area like system optimization, then you'd fill in the baseline—what's now—and the proposal, which is what you expect when you're done with the work. You do not fill in the end of the project. That only gets filled in when you report your LCOE after you've finished the project.

There will be differences. Some things will turn out better, hopefully. Some will not turn out as well. But we want to know at the start what you have to begin with, what you expect to accomplish, and then at the end what you actually did accomplish.

For the initial capital cost—this is confusing, I know, because it's not the way most people are pricing things out, but you want to show—in the end you want the price at the bottom to be exactly what you expect the customer to pay. So, you don't want to include your profit or development costs spread out over—that goes in the "other" that is the difference between what it costs you to manufacture the turbine and what the sales price is to the customer. So, we don't ask you to spell out what you expect as a profit because it's—there's a myriad of expense under "other," including keeping the doors open and answering the phones, and advertising, and et cetera, et cetera.

We used to get really, really amazingly optimistic installed costs, but we now have pretty good data from a lot of installers about real installed costs, so don't be ridiculous with us because we'll know.

Let's see. Balance of system. So, this is everything other than the turbine and tower, everything that gets planted into the ground or assembled onsite or created onsite. Your foundation, the installation work, the electrical infrastructure, et cetera. And again, we have an "other" thing on the bottom, and that "other" category should be used to put the difference between what all of this stuff costs and what the customer pays. You would probably put your installer markup in there, for instance. Maybe not. But that's the way I would do it if I was doing this. And again, the baseline, the proposal. Not the end of project until the project's over.

Oh, yes. This always throws me. I've got here "in the past" only because I'm not allowed to talk about the future. And I'm always going "What am I trying to say here?" Yeah. The RFP will specify an average wind speed—a distribution usually, really—and a sheer, and a standard height. Now, you're more than welcome to say, "Oh, well, I'm gonna—I'm doing a tower modification, so I'm gonna make super tall towers affordable." So, instead of, say, the 30-meter normal tower height that you give in the spreadsheet you want to put in a 50-meter tower. That's great. You can enter a 50-meter tower. It will actually—the spreadsheet will recalculate the wind speed distribution based on the height of the tower, and the—whatever it is—probably 6 meters at 30—or, 6 meters per second at 30 meters, and then it'll calculate your annual energy based on—[coughs] excuse me—based on that different tower height. And that may be the difference between your baseline and your proposal, if for instance that's part of your proposal, is a taller tower.

You have to provide the power curve. And you want to let us know how you got there. Is it measured because you've built some of these? If it's the baseline, you may be working from an existing product. If it's—or you may have built a prototype. Or it could be just calculated at this point. You may have built a FAST model and designed a rotor system and played with it and you're pretty comfortable you've got a power curve from that.

Please don't exceed the Betz limit. We're not going to believe you if you do, and I've seen plenty of proposals that do. Credibility gets low real quickly.

Again, so often people get this wrong, so you'll notice that this is like the third time I've said: Provide the baseline as it is today and the proposal as expected to be after the improvement. Don't do the end.

If you're projecting a significant increase in performance, justify that—not in the spreadsheet but in your proposal. We really want to know what the basis is. If you're not increasing the swept area you really have to let us understand what's going on that makes a difference. Clearly. Not just "We're going to improve it and it's going to be 20 percent better."

If you're doing calculated you definitely have to do adjustments for things like controls because there are losses in the controls. If you're doing measured, controls losses is already in there.

Oh, yeah. Here's the height that we talked about. There may be a grid availability in the RFP. We just tell you to take two percent or something like that. So, pay attention to that stuff. We appreciate it if you follow those instructions.

O&M costs and levelized replacement. Turbines do take maintenance. If you project zero O&M costs for the life of the machine we're not going to view that as being very realistic. If you call out an annual maintenance, then put in the money for an annual maintenance, including whatever costs they're going to be—travel expenses, whatever. I mean, obviously travel expenses vary enormously but they're never nothing.

If there's a—if, say, you're—you have a main bearing that has a predicted life of 12, 13 years so you plan on replacing them every 10 years, that does not go into O&M. It goes into levelized replacement. So, anything that's expected to be replaced periodically other than brake pads and oil, which are consumables and do go into O&M, should go in the levelized replacement. So, you just have to enter the cost and the mean time for replacement and the spreadsheet will calculate the annual cost for you. The spreadsheet's really a useful tool for you in looking at all of your cost of energy, not just for these proposals. I think it's a gift to have that out there for people. And you can customize it any way you want to get more granular in the cost of the equipment or whatever, but it's a nice framework.

OK. I think I'm done. So, we're back to questions.

Moderator: Great. There are a few here. One from Erica. "For pre-prototype do you expect to see baseline and proposal columns both filled in?"

Male: Not necessarily. In fact, I've seen and been happy to get pre-prototypes with the baseline being PV or another turbine from another manufacturer that they were going to be competing with, and then the proposed was what they thought they could get to with their design. So, pre-prototype, we can—it can be a lot more flexible. But give us some information so we know that you've thought through this.

Moderator: And then, certainly one other key, especially for kind of the pre-porotype and the prototype testing, again, we know that any costs at that point are initial estimates, educated estimates, but certainly not final costs because you're not there in the design process yet. And —

Male: Absolutely.

Moderator: —that's understandable. Provide documentation to the extent that you can, that you have it. To a degree, the more information that you have, the better, just because it clearly demonstrates that you're thinking through the steps.

One of the things that this spreadsheet does—and you can see it's pretty detailed—we're not trying to inflate costs or anything of that nature, but we're trying to give a real, realistic picture to the cost of the technology. One of the guides for us from the proposal standpoint to understand what kind of innovation, what the final outcome of the innovation is based on what you've done or what you propose to do, or what you hope to do, but I think it also is a pretty good indicator of where you are in the competitive market and your kind of path to a real market. And so, if you are a turbine and you go through this and you're doing a component innovation but you're still three times your competitor, well then, even though your component innovation might be a very viable path forwards, if you are not reducing your costs significantly through that process, that might not be the best investment that you have. And so—and it might be that what you need to do is a full system optimization to be able to cut your costs more significantly, and taking a stepwise approach might not be the best way to do it.

So, to a degree, through this kind of more formalized process we're holding all of us to the fire so that we can make sure that what gets funded has real impact, but that we're also investing in technologies—and you're investing your time and energy into technologies that have the best path to marketability.

Great. Any other questions for Robert or other kind of additional considerations around these kind of additional CIP considerations?

All right.

Male: Here's—there's a—Jasmine put in a text. "What is your feedback process after the RFP? Do you provide a detailed analysis if we do not win?" Yeah.

Moderator: Yes. OK. Yeah, so I can dive into it. To a degree—and Kyndall, I'm not sure if you're still on, but to a degree it's a two-step process. So, within the formal RFP process you submit the RFP, we do this evaluation, all the scoring, and it's a pretty detailed process. And then we let the—we kind of announce the proposals that we have selected for contract negotiation. We—and then we let all of the other companies know that they haven't been selected. It's a little bit of a kind of time delay here. There is the potential that some of the awardees will not go through the negotiation process. That hasn't happened yet but there is that potential. And then, we do do formal debriefs if an entity wants a formal debrief as part of the formal review process. And you—Kyndall will let you know—if you're not selected for negotiation you have that opportunity to ask Kyndall to do a formal debrief.

The kind of unfortunate thing or the time-complicating thing is that we can't begin those debriefs until after the other awards—until all the awards have been finalized, and that's kind of one of our legal requirements from a contracting standpoint. So, it does mean that there is a long period of time and potentially a too-long period of time between when the contracting—the selection is made and the contracting process with all the selected awardees is complete and potentially the new round of CIP going out.

One way that we can to a degree get around this—or not get around it, but if you do not want a formal review of your proposal and you state that you do not want a formal review of your proposal, then your proposal is no longer being considered and we can talk—now, again, we can never talk about what the next RFP will be, if there is even a next RFP, but we can sit down and talk more about where your technology is and what might not make a better proposal per se, because we can't really talk about the proposal, but we can talk with a company about where they might be from an innovation standpoint. And so, although we can't go back and say, "Well, you got a 22 on this topic because you didn't answer this question," we can provide support that will lead to a better proposal. And that can happen basically as soon as you state that you do not want a formal review.

Is Kyndall on still? I'd ask her to comment. No, she's not on.

So, that's—it's a little funky, and part of that is just the legal process. Conceptually, until all the awards are finalized, a company could contest one of the awards, and therefore we can't dive into the kind of giving feedback to people who were not selected until all the other awards have been signed, the contracts have been signed. So, a long-answered question to the "Do we provide feedback?" We try to.

Any other questions? Any other questions? Or comments? Happy with comments.

All right. Well, thanks all. I guess we'll move ahead and close up a little bit early. I know we all have work to do. Thanks again —

Male: Ian?

Moderator: Oh, go ahead.

Male: Yeah, let's just make sure that there aren't any requests for future changes in the program, things that we don't cover that they'd like to see covered or whatever. Format changes. Whatever.

Moderator: Yep. So, building on that, we're always looking for improvements in CIP, additional topical areas that we could include, or ways that we can streamline the process getting over those moats, as they were. I mean, again, some of them are—we won't be able to change. Some of them are out of mine or Robert's or Kyndall's direct control. But to the extent that we can we try to work around those and make it as easy as possible for all of you.

Clearly, our e-mails are readily available, and so if you have ideas or just want to talk about things more, we're more than happy to have those conversations. And then, as I think we've said a couple of times over the course of the day, if you would like to have a detailed conversation, a more detailed conversation with any one of us about your technology and where it may or may not fit into CIP, now is the time to do that before the RFP goes live. And so,—assuming that there is an RFP but—the RFP goes live. Now is the time to have those conversations.

So, more than happy to do that and the process would be to reach out to basically any of the speakers her, but myself or Robert or Kyndall, and say you would like to have a conversation about something that you're thinking about, and then we can sit down and do that, whether it's a couple—whether it's a half an hour, an hour, or even if it's a more detailed conversation. We want to make sure that we're not wasting your time in pulling together a proposal that missed because you thought you should be submitting it under component improvement and it really fits better under an optimization or a testing.

OK. Great. Thank you all. Again, hopefully next year we can do a combination of in-person as well as Web, but at least we're not going to be working from our basements a year from now, with fingers crossed. Certainly reach out if you have any other questions or comments, considerations, or thoughts in regards to CIP. We're always open to ideas.

Everybody stay well. Thanks again to the presenters and thanks again to all of you for taking part in today's session.

Male: Thanks.

Moderator: Take care, everybody.

[Crosstalk]

Moderator: Yep. We will be—again, we will be making these available. We're hoping to have them done by the end of the month. So, you'll have it in kind of the beginning of January.

Mike Bergey. Can we unmute Mike Bergey?

[Crosstalk]

Male: Yeah. Can I get a comment in quick?

Moderator: Of course you can.

Male: OK. So, given the incoming administration I'd suggest a new topic area that would be applications-oriented support if you come in with another government agency—a military or foreign assistance or Department of Interior or some other agency and they want to do some sort of a proof of concept application of distributed wind, and just have that as a topic area.

Moderator: So, let's pull on that thread a little bit, Mike. So, the idea would be DOD is looking for cost share to deploy a wind turbine in the Sudan, and so they—and so, the idea is this is a proof of concept. They want kind of rigor behind it. They're willing to put some cost share into the deployment space or something like that. And the idea is it's kind of—I don't know how much of it is new technology but it's certainly kind of innovation or proof-testing an application in a unique environment?

Male: Right. It's got some possibility of replication. So, a forward-operating base application and agricultural productivity, water pump—wind electric water pumping or USAID. That sort of a thing where they—for example, USAID program managers have $100,000.00 discretionary, they can reprogram up to that, but that may not be enough to fund a project to try something out. That would be an example.

Moderator: Yep. OK. OK.

Male: Needs a little thought to —

Moderator: Yep, a little noodling.

Male: Right.

Moderator: OK. But definitely noted. Definitely noted. So—and part of my thinking is—my first take is always can we fit it—will it fit within one of the current topical areas?

Male: Well, that would be fine if you just then in your instructions, your examples make sure it's clear that that's within the bounds.

Moderator: OK. Oaky. No, certainly. Certainly. OK. Great.

Male: That was it. Thanks. Thanks for a good meeting.

Moderator: Certainly. Good. All right. Thanks, everybody. And again, don't hesitate to reach out if anything comes up. We're here to help. Take care, everybody.

[Sounds of video conference hang-ups.]

Moderator: We still have you on, Rob. The diehard. Rachel, are you on? Are you there?

Rachel: Yeah, still here. Are you ready to close it down?

Moderator: Yeah, I think we're ready to close it down. If you could copy all of the notes—I don't think we had any Q&A.

Rachel: We had four. I copied them. We had great chats. There weren't any chats that I could find in the breakout sessions. But I made a lot of notes. It's OK but… So—and then there should be—I recorded this and I'll work on downloading that and getting a recording for you with notes and everything. And attendees and all that.

Moderator: OK. Yeah, we had no [inaudible]. That's an issue. How do we do that? I'm not sure.

Female: Wait, you had no what?

Moderator: We—I was looking at the questions and how there were comments that we had only one female panelist. So, we need to figure out how to get more females into the CIP ranks. Very good point. Very good point.

Female: Yeah. Very good.

Moderator: Yep. OK.

Female: OK. I'm going to turn it off. Thank you, Ian.

Moderator: Great. Thank you.

Female: All right. Talk to you later. Bye.

Moderator: OK. Bye.

[End of audio]