NIJ is committed to improving timely, public access to the results of NIJ-funded research. We are developing a policy requiring that peer-reviewed journal articles and their associated data be deposited upon publication and instituting the use of persistent digital identifiers for all investigators, institutions, awards, publications, and data sets. The purpose of this planned policy is to make research findings available to the public and encourage more replication and reuse of data than is currently possible. This webinar will discuss the draft plan and allow attendees to share their thoughts and suggestions with NIJ staff and leadership.
Comments on the draft plan are due November 29, 2024. See the Federal Register announcement Request for Comment on NIJ Draft Public Access Plan.
Transcript
STACY LEE: Welcome, everyone, and thanks for joining us today for the Expanding Public Access to NIJ-Funded Research webinar, hosted by the National Institute of Justice. It is my pleasure to introduce NIJ's Director, Dr. Nancy La Vigne.
DR. NANCY LA VIGNE: Greetings, everyone, and thanks for joining this webinar on NIJ's exploration of expanding public access to the data and research that we sponsor. I am very excited to be co-hosting this with my colleagues because it's really important to us at NIJ that we take the time to listen to the field and engage with our stakeholders, especially when it comes to making what feel like pretty big changes to policy and practice. But what I want to reference here, and we’ll be repeating this message early and often, is that while this may feel like they're sweeping changes, they’re perhaps not as big as what meets the eye or the ear. What I mean by that is, yes, we're looking to follow best practice, following in the footsteps of other federal agencies in making access to the data that's produced by research that we sponsor, as well as the research findings, available to people more easily through both open access to publications as soon as they're published, as well as access to the data that was used to produce the work that's in those publications. I know that that can give people pause for a variety of reasons, and we're interested in hearing what those reasons are. We understand that they might have cost implications, they might have implications for feelings about what's proprietary at what point in the process you are in developing articles and developing findings based on the work that you've led.
But what I want to underscore is that this is not making any real changes to the archiving policies that we already have in place. Our archive policies are very stringent in that we make sure that we're never releasing public data that can compromise people's identities or their privacy. We have a whole array of ways we do that. And we do have a partnership, of course, with the National Archive [on Criminal Justice Data]. When we've been on listening tours talking about expanding public access or open access, we've heard a lot of reservations. “What if I have qualitative data? It could be identifiable.” These are issues we have long navigated. We will never archive information that compromises privacy. Instead, we come up with solutions like aggregating the data, summarizing the data, and/or making access to the data very restricted to other researchers that have to make the case of why they're going to be using the data, and how they're going to be using the data, and how they won't be using the data. So, I just wanted to start with that message. We'll be, again, sharing it with you over the course of this webinar and also answering your questions when Q&A time comes. But I wanted to start there.
And I also wanted to acknowledge Stacy Lee and Mike Lambert who are behind the scenes supporting this webinar and almost every webinar NIJ does. They're really wonderful colleagues and we appreciate them greatly. And then we have a number of colleagues that have joined this webinar, some of whom will be speaking. I want you to be able to meet each and every one of them so that you can get to know them by name and face and role, because another thing that we're trying to do here at NIJ is be more accessible to you, so you know who we are and what we do - to put faces and names to the agency. So, with that, I'm going to turn it over to Greg to introduce himself, and then we'll bounce it to Kyleigh, Lee, Liz, and Erin, and then it will be back to Greg to launch the webinar.
DR. GREGORY DUTTON: Great. Thank you, Nancy. Hello, everyone, my name is Greg Dutton. I'm a Program Manager at NIJ. I manage programs in forensic science R&D and graduate research fellowships, and I have a long running interest in open science, so you can't keep me away from this topic. That's why I'm here.
DR. KYLEIGH CLARK-MOORMAN: Hi, everyone. I am Kyleigh Clark-Moorman. I'm a Social Science Research Analyst here at NIJ. I work in the Office of Criminal Justice Systems. Thanks.
LEE MOCKENSTURM: Hi. I'm Lee Mockensturm. I'm a Writer/Editor at the National Institute of Justice, in our Office of Communications. I also manage nij.ojp.gov and crimesolutions.gov and I work a lot with the NCJRS or OJP Virtual Library.
DR. ELIZABETH GROFF: Hi. I'm Elizabeth Groff. I'm a Senior Advisor in the Office of the Executive Science Advisor at NIJ.
ERIN PARTIN: Good afternoon. I'm Erin Partin. I'm a Data Scientist at NIJ and I serve as NIJ's Data Officer. Happy to participate and help answer any questions that folks might have.
DR. NANCY LA VIGNE: Fantastic. What a great team. And this is just a small slice of all the staff and the expertise we have at the Institute. I'm going to pass it on to them to launch us into the content of this webinar. Thank you.
DR. GREGORY DUTTON: Thank you, Nancy. Thank you, everyone, for joining. Why are we here? We want your input. So, if you're on the webinar, I think you know that we have a Draft Public Access Plan out on the Federal Register that's open for public comment. It's open until November 29. We encourage you and everyone you know to comment. We have a QR code there. I'm also pasting into the chat the URL to the Federal Register notice. So, as we go through the webinar, you might find it convenient to pull that up and refer to that as we go through. The plan is up for comment until November 29. We're hoping to finalize a plan by the end of the calendar year and the plan would likely not become active until 2026. So, we want a final plan that serves our community, and if you're on this webinar, you're probably a part of that community.
Today, what we'd like to do is discuss the motivation for this effort overall and then step through the details of the draft plan. My colleagues will do that. And give you an opportunity to ask questions. So, feel free, please, to enter questions into the chat and then, at the end, during the Q&A, we'll address your questions.
First, we'd like to get a sense of our audience, so we have a poll question. We'd like you to answer the poll. “Have you received or applied for NIJ funding before?” Yes, no, or not applicable. I put not applicable because I'm not eligible. So, we're getting mostly yeses and some nos. So that's good. I think we have a wide range of grantees, non-grantees. We want to have a broad representation.
Thank you everyone for answering. So why are we focusing on public access? Well, for one thing, we all know that science builds on prior findings. And recent advances in digital resources and data have led to a movement known as open science to try to accelerate the process of science by making the results, publications, and data available to a broader range of people earlier in the process.
Also, this fits into NIJ's mission. NIJ is an applied science agency. So that means we really are focused on serving a mission and that's criminal justice. NIJ focuses on dissemination of knowledge. We recognize that for us to serve that mission, the results of our research, publications need to reach the audience that can use it to affect policy and practice. Thirdly, the taxpayer has an interest in knowing if their taxpayer funds are being used responsibly. So, we want the public to be able to see what we're doing and weigh in on the value of that. And then, finally, like Nancy mentioned, we're not doing this in a vacuum. There was a 2022 Office of Science and Technology Policy memo from the White House directing federal science-funding agencies to make their publications freely available to the public upon publication. So, embarking on this puts us squarely in the mainstream of other federal science-funding agencies. And if you have insight into that, you'll see lots of other agencies doing very similar things now. If you only look at NIJ, this might look like something coming out of the blue, but it's really not.
So, we've been trying to think about the motivations and stakes that the different people in our community have in this. We think there are benefits to everyone and certainly there will be challenges too. We want to consider that, and we encourage you to weigh in on what's in it for you? What's in it for the various stakeholders? What's in it for practitioners? We think this is key. Practitioners really need access to the latest research without barriers, without paywalls, for example. A lot of our practitioners, many agencies that don't have the subscriptions to journals that, say, large research institutions have. So, they might really want to read the latest NIJ research, but they might hit a paywall.
What's in it for researchers? This is actually a positive thing for them. This will give them wider audiences for their work, which they want. Possibly more citations. It's not quite clear in the research whether open access does lead to more citations, but it certainly leads to more downloads and page views of journal articles. Easier access to other researchers’ data for reuse and their own data cited as a product of the research. Just like journal articles, data now is recognized as a primary research product. So, if you can have your data cited, that's a bonus.
The public? Again, the taxpayer deserves to know what we're doing with their money and if they should continue to fund more research in NIJ. So, we want them to have full knowledge of what our research has produced.
What's in it for us? We’re really excited how this can help us learn how effective our programs are and to manage them effectively.
As we go through the talk, I’d like you to keep in mind the FAIR data principles. These are some open science principles you might be familiar with. A lot of these things apply to some things we're talking about. Certainly data. Also, publications. So, these are things to think about.
First, is it findable? Does the publication or the data that we're talking about, does it exist somewhere digitally, and can I find where that is? And where people look to search for things may vary depending on their field of study. It's important that we make sure that they're able to find the things that they need. Secondly, if I can find it, can I access it? So that might mean, is it behind a paywall that may be a challenge for me to pass, or that also can mean is it accessible for people that may have challenges in reading products in certain forms? Interoperable. Is it in a format that other users can use effectively? And then, finally, reusable. This mostly applies to data. Does it have enough information and metadata for it to be effectively reused? If not, then it's not as useful to others.
There were three public access plan elements that we put forth in the draft plan, and these are really the key, core concepts, the three broad goals that we have. How we're going to meet those goals and detail is what we're hashing out through this whole process. The elements are first, that we'd like to use persistent identifiers and metadata to help with findability and reuse. Secondly, that peer-reviewed publications will be made freely available to the public without delay. That's really a core element. And then, lastly, that the data that supports those peer-reviewed publications will be made available at the time of publication. As we go through the webinar, my colleagues are going to discuss these three elements in greater detail. And I'm going to pass it off now to Kyleigh.
DR. KYLEIGH CLARK-MOORMAN: Thank you so much, Greg. Hello, everyone. I'm going to start with a conversation about persistent identifiers, which are also called PIDs. A persistent identifier is a unique identifier for research information—that can include publications, data, researchers, institutions—that is persistent, machine processable, and follows metadata schema. In practical terms, this is typically some type of code that allows you to access associated data about people, places, and things. Our goal here is to promote scientific integrity by requiring the use of these PIDs. It allows for the identification of entities and interconnections between researchers, organizations, and outputs. And it also enables the findable portion of fair data exchange principles by keeping everything all in one place and connected.
So, to lay out our potential changes, we thought it would be a good idea to compare what we already do in the current policy with what would be the potential policy under consideration in each of these different areas. Currently, for PIDs for researchers, we request these during application, and a lot of people submit them usually in the form of ORCID IDs, but the change would involve requiring the submission of PIDs for all senior grant personnel in whichever form they prefer. There are other PID services other than ORCID IDs, but we do assume that most of them will probably be in that format.
Currently, we also collect digital object identifiers for publications during normal reporting or data collection, and the change would be that we would now require the submission of PIDs for peer-reviewed publications from NIJ-funded products. And that would be before the end of the grant period, which again is not a big change from what we already do.
In terms of data, data that is deposited in our National Archive of Criminal Justice Data or NACJD at the end of a grant period already receives a DOI. Again, we would just require PIDs for publications and data. Those DOIs could come from NACJD or another data depository, depending on what the researcher is going to use.
Then, finally, in terms of PIDs for NIJ products, we currently do not have DOIs for awards, but we do have a landing page on the NIJ website that shows publications from each award and things like that. But we would move to assigning PIDs to awards, final reports, and other NIJ publications as possible, again, to increase that interconnectedness.
I'm going to go over what this looks like in practical terms on the next slide. But before I do that, because it's going to involve looking at my ORCID ID page, we do have a poll question for everyone, which is whether you currently have an ORCID ID. And I'll give, again, about 30 seconds before I look at the answers. All right. Do we have those answers?
STACY LEE: We do. We have about 34 people who said “yes” and about 37 who said “no.”
DR. KYLEIGH CLARK-MOORMAN: So, it's about split. Thank you so much, Stacy. It's good to know that some of you are going to be familiar with what I'm talking about but also that some of you are not because ORCID ID is really a great service. I should say this is not an endorsement of ORCID ID. There're other services that you could use for this. But, again, this is the one that we typically see in applications, and I think that there are some advantages with data linkages because it is so widely used and well-known for this service. So, actually, when I decided to pull these things up for this presentation, I tried to get into my ORCID ID, and I was locked out because it was an old email and then I also saw that it hadn't been updated in a few years. So, I went through their customer service, which was super helpful. They got me reconnected in a couple days. And updating my page was extremely easy. I did some linkages. I did some manual imputation. But just encouraging you to look at this service if it's something that you're interested in.
When you go to somebody's ORCID ID page, essentially, this is what you would see, some of these screen grabs. Again, it's a way that you can connect everything that you're doing as a researcher. When you first go on, you'd see a record summary. You can see some affiliations, some works, some education and qualifications, and there's a number of other categories that I didn't show here because of the limited space. But if you would go down further on the page, you'd see more information about me and what I do and what I've done in research. You can list all of your work. These are just examples of a couple of things that I've done. You can see the title of the work. You can see who else was a contributor to the work. You can see those DOIs for the publications if they do exist. You can also see where I've been employed. Obviously, I'm working at NIJ now, and you can see that even our organization has a persistent identifier there listed under the ROR. It's just a way that you can really see these interconnections between what someone is doing and the work that they've put out and who they work for, who they work with, and other affiliations.
Just to talk about like the advantages of why you would want to use something like that or why we would like to require something like that is first, it allows for the identification of these linkages between researchers, organizations, and outputs, and that really encourages findability. It can also possibly increase usage of these things because they're all connected and easier to find. If you know a researcher is researching one topic, maybe you can go on to their ORCID ID page and see all the things that they've written on that topic.
Additionally, it's common for institutions to require biosketches when applying for funding or during regular reporting, and that means that you have to identify your publications, your funding, your affiliations. So, having an up-to-date ORCID ID or other PID service can help with the easy transfer of information. And, again, a lot of these things are interconnected. So, if you publish a journal article, you can submit your ORCID ID, and it'll go automatically on your profile. And some agencies even require an ORCID ID to automatically fill in this information. For example, the National Institutes of Health either currently requires or is going to require use of a common form. And that's linked to someone's ORCID ID. They can put in their ORCID ID information, and it'll pull the information in automatically. So, again, it just kind of makes these things easier when it is time to report all of these requirements, like publication funding and affiliations. And that's all I've got for PIDs, but I'll turn it over now to Lee to talk about publications.
LEE MOCKENSTURM: Hello, everyone. Thank you, Kyleigh. So, our goal with publications is we want to make sure all the results of our research are readily available and that we, NIJ, know that they're out there. Our goal is to have this done freely and as quickly as possible without delay. This addresses the findable and accessible portion of the FAIR document exchange principles. If we can collect all of the information related to NIJ's awards in one place and have it readily findable and accessible to anyone, and that includes accessibility for people with disabilities, it really increases the reach of what NIJ does and lets NIJ know what that reach is. So, we're really trying to connect all of the publications that result from funding, whether they're peer-reviewed or not with the NIJ project itself.
So, you go from “we gave out this money” to “this is the resulting research” and “this is the knowledge that we've gained.” A lot of what Greg and Kyleigh talked about is going this route and making things really accessible and open access is really going to increase the number of people that can see it, and that's increasing the number of researchers that see it and also the number of practitioners that see it. Also, I'm talking about making these available in machine readable format so they can be ingested and read by other systems as well and completely accessible for anyone with a disability.
So, currently who can access the peer review article is left up to the publisher or the author. Whether it's in an open access journal, whether an author of their institution has paid for open access, it's really all outside of NIJ's hands. We also don't have a good single way for our researchers to let us know when they've published an article. We've made some big strides in doing that. We do sort of monitor the ecosystem. We have an online forum for people to submit to. But none of those things are yet requirements. So, it's hit or miss. And, hopefully, more hit than miss, but we'd really like to make it a systemic approach. All of our publications are also entered into the NCJRS Virtual Library or the OJP Virtual Library. And that's all done manually, which is a process that's really prone to duplication and errors. What we're looking to do and our plan, and this is in the draft that's available in the Federal Register is to require all our grantees to submit their peer review publications to PubMed Central.
PubMed Central is a fantastic resource. There's a lot of infrastructure behind it that NIH has developed over the years. And they make that available to other federal agencies, so we don't have to duplicate all of that work. One of the things they also allow us to do is we'll be able to take the publications that our researchers enter into PubMed Central and move it over into the NCJRS Virtual Library seamlessly, and, that way, all of NIJ's work from the past and moving into the future will all be available in one place and we can capture that information and connect it to the award that the work was done under. We also hope that this can be a way for us to enforce some compliance on our grantees because we'll be able to see when they upload it. It'll be a requirement of the grant and not just something nice for us to know.
The real core of everything we're going to be doing with publications is really the use of PubMed Central. So, we wanted to get a feel at least for the people in the audience, whether they're familiar with PubMed Central. Have you ever used PubMed Central? And that would be either that you've submitted something to PubMed Central, you've searched PubMed Central, you've downloaded from PubMed Central, or just kind of a general familiarity.
STACY LEE: We have about 29 people who have said “yes,” and 33 have said “no,” and the others did not answer.
LEE MOCKENSTURM: All right. Great. Thank you all for participating. So, just a little bit about PubMed Central for those of you who aren't familiar with it. It was developed by NIH, and it's managed by the National Library of Medicine. It is really a place for the public to go see the results of federally funded research that's freely available. Primarily, you're going to find articles there that are the author copy of an article that has been accepted for peer review. The source of the information in PubMed Central is either the PI themselves can upload the information, someone who is a representative of the PI can upload, and then the PI would confirm and approve that that's the accurate information. PubMed Central also has a wide array of primarily health-focused journals that they automatically accession data from.
So, if you're an NIH awardee or you're an awardee from the CDC, if you're publishing in a journal that is commonly accessed in the health profession, that information may be already pulled into PubMed Central directly. For NIJ, for our researchers and our PIs, primarily the information is going to get into PubMed Central by you all. But they have a very robust, a very well used, well-tested system for uploading information. Another thing that they also have is lots and lots and lots of tutorials, including video and slideshows that will take you all the way through to providing a title. They show you how to link your title to the funding that you received, how you upload your files, how you upload the exhibits and the tables.
And then there's a lot of confirmation as to who you are to make sure the person uploading it has the permissions to upload it and is associating with the right grant. Then finally, there is a final review where someone in your organization, typically the PI will go in and say, “Yes, this is exactly what it says it is.” We're still going to work through how NIJ staff will be involved in this to make sure when someone says, “Yes, this is the result of an NIJ award.” We can go and look and confirm that, yes, this is the result of an NIJ award. We can also make sure the award number and all the information is correct. If you're interested at all, some tutorials listed on the screen there. (Slide 17) You can also just go type in PubMed Central tutorial into any search engine, and they will get you right to this page.
So that was it on publications. I hope to see your questions. And again, well, I'll make another plug for please respond and submit questions to the Federal Register announcement and the email that we provided.
DR. ELIZABETH GROFF: Thank you, Lee. At this point you've heard more details about potential changes in the use of digital identifiers and in making publications freely available. Now, let's cover the potential changes to data requirements. With these changes, NIJ seeks to encourage secondary data analysis by requiring that grantees make the datasets used to produce the findings in peer-reviewed publications available upon publication of the article. In addition, that the grantees deposit these data in a publicly accessible repository. Finally, that the grantees provide sufficient documentation to make the data usable by others. To try to make this clearer, there are three important components when thinking about data. The timing of the data deposit, the scope of the data that needs to be deposited, and the sensitivity of the data involved.
I've put a table here and a sample three-year grant timeline to illustrate what I'm talking about. You'll notice that the blue column talks about the current policy. The orange column or yellow, depending on your monitor, is the potential policy. And then I've got the project end date. The blue line represents the current policy, and the orange represents the potential policy. Let's go through the current policy first. As far as timing, grantees submit their project data at the end of the project to the National Archive of Criminal Justice Data. As far as scope, they deposit all the project data that was collected using NIJ funding and metadata to support those data. Related to data sensitivity, some data don't involve human subjects and are able to be made freely available immediately. Other datasets have varying levels of privacy protections necessary. As the Director mentioned, NIJ has a long history of archiving data with sufficient human subjects’ protections.
Moving now to the potential policy, let's say the grantee publishes an article during the grant period. That's what this potential data policy is all about. As far as timing, the data deposit happens at the time of publication. As far as scope, the data is the subset of data and relevant metadata necessary to produce the findings in the publication. And then as far as sensitivity, necessary protections for sensitive datasets, such as qualitative data that carry a risk to human subjects will be applied using our current techniques for archiving. For example, we have a restricted data repository that's available.
This slide reiterates that the prospective policies are consistent with our current requirements. Essentially that our grantees provide data and appropriate metadata, and that we establish a level of access that's necessary to protect confidentiality and privacy, and we have different ways to achieve that.
Turning now from data to what's next. We've been conducting stakeholder engagement at conferences and now we’re holding this webinar. But our primary conduit for receiving comments is via that Federal Register announcement. In that announcement, we ask specific questions and request responses that detail challenges, opportunities, and benefits that you see from the proposed policy changes.
Our plan is to take all your comments right after the announcement closes, which is on November 30th, and analyze them and then use them as we consider what our final policies will look like. We'll be implementing any new policies via revised award conditions and new mechanisms for monitoring grantee compliance. We anticipate the new policies will be effective for fiscal year 2025 awards.
This concludes the content of the webinar We're going to address any questions that have been put in the Q&A. Again, we request that you submit any comments that you have to that Federal Register announcement so that it can be part of the data that we're collecting.
So, I'm going to switch hats now and start reading the Q&A. And any of the panelists who have an answer, can feel free to chime in as we go. All right. First question, is the ORCID process free?
DR. KYLEIGH CLARK-MOORMAN: I should have mentioned it that yes, it is a free service. There are institutions that can pay to be members. But as a researcher, you can join the service for free.
DR. ELIZABETH GROFF: Thank you. A multi-part question … must be from an academic, right? “I hope to hear one, how DOJ will protect tribes when researchers tend to highlight only the negative aspects of tribal crime/victimization, thus contributing to negative stereotypes of Native people. And two, how DOJ will incorporate culture-specific responses to tribal-based crime. And three, how NIJ will protect data from tribes?”
DR. NANCY LA VIGNE: There's a lot to unpack here and it's not all related to this particular initiative. It has to do with issues of data sovereignty that go beyond this potential change in policy but certainly are relevant for tribal communities in which research that's sponsored by NIJ is conducted. I guess the easiest answer to this is to say that we are clearly aware of issues with data sovereignty in tribal communities, that we are participating in the tribal listening session in just a few weeks' time and have a specific session where we're sharing our thinking about our past investments in tribal research and implications for future investments in a way that I think mirrors what we're doing here, which is really trying to be a lot more intentional in our public engagement and listening to the field.
So, whether there are different policies that apply to research conducted in and on tribal communities moving forward as it relates to this policy remains to be seen. But I do appreciate the question and rest assured we're aware of these issues and are grappling with them and in close engagement with the vested stakeholders on the topic.
DR. ELIZABETH GROFF: That was a much better answer than what I could have provided. Thank you for that, Director. The next question is “Criminology, the journal, was looking into requiring datasets be submitted with journal submissions. Is this a must happen in DOJ? If is it a must happen and not a might happen, qualitative researchers will have a big problem with this.” Erin, did you want to take that since you're here and it is your area?
ERIN PARTIN: Sure, yes. So, in response to your question, yes, the idea would be for any peer review publication, the underlying data would be required to be archived at the point of publication, not simply at the end of the project period. This would include qualitative data. This would include the identified data. Current archiving includes a robust back and forth with our partners at NACJD about appropriately ensuring that the data would not allow for the disclosure or identification of any human subjects. Every research project is different. Every project would need a different degree of scrutiny. Those are issues that we are happy to work with you through on specific questions throughout the process.
DR. ELIZABETH GROFF: Greg, did you want to add something?
DR. GREGORY DUTTON: Yeah, I wanted to chime in. I read this question as maybe talking more about the timeline for when data would be expected. This sounds similar to the American Chemical Society. ACS has dozens of journals. And they, I believe, are also starting to require data to be submitted when you submit to the journal. So that it's available for reviewers to look through the data themselves. Now, not all journals are doing that. More and more are. From our perspective, if the journal requires it, that makes this easier because then you're going to have to have the data available somewhere. But I think that's not something that we would recommend, necessarily. But we're interested in having the data available when the article is published at least. So that's what I thought this is getting at, more of the timeline.
DR. ELIZABETH GROFF: I think it's great to try to be as broad as possible in our answers since we're not letting attendees verbally comment or elaborate. Attendees could always clarify their questions by elaborating in another chat or another Q&A question. The next question is, “Will PubMed accept full published articles? I only have published pieces, not the original drafts.”
LEE MOCKENSTURM: Yeah, so PubMed Central is really for submitting the author manuscript. And we're talking about moving forward. This isn't something we'd be asking grantees or PIs to retroactively go back and add that data. There are other ways that content winds up in PubMed Central or on PubMed in general, and other ways that we try to put things into our own virtual library. But we wouldn't be asking you to take your final product and retroactively put it into PubMed Central. PubMed Central is really designed for author manuscripts only.
DR. ELIZABETH GROFF: Thank you. Okay, next question. “Does the data archive need to include every variable, or should it just include the variables used for the analyses?” For the publication, so this is the dataset that you would be providing if you publish an article, then it would just be the variables used to produce the findings in the paper.
DR. NANCY LA VIGNE: But worth clarifying that as an NIJ grantee at the end of your grant, you're expected to archive all of the data.
DR. ELIZABETH GROFF: Okay. Yes.
ERIN PARTIN: Yes, that's correct.
DR. ELIZABETH GROFF: Absolutely. Thank you. Let's see. Some more public access ones.
DR. NANCY LA VIGNE: I think there's a question about leniency and filling qualitative data requirements because it's intensive to analyze the data.
Well, what we're asking for is if it's at the point of publication, you would have already analyzed the data. And the goal here is to have the data available at the time of publication with the caveat that we understand that qualitative data would need to be treated differently than quantitative for obvious reasons.
DR. ELIZABETH GROFF: Thank you. Another question now, “Peer review articles are often published well after a project ends. Is NIJ intending to allow additional grant years and increased funding to account for this?”
Our grantees will have already archived their data at that point with the National Archive of Criminal Justice data. So, no, we're not going to continue to monitor our grantees for things that happen after the end of the grant period at this point.
DR. NANCY LA VIGNE: However, we do require that they include the award number in all publications, and we do track all of those publications because tracking impact is important to us.
DR. ELIZABETH GROFF: Okay, next question. “As a correctional entity and a data source, we have a standard data sharing agreement with all researchers, which specifically does not permit secondary dissemination of our data. We have not at this time had any researcher let us know that our provided data would be shared in any way. We will have to supply a greater scrutiny to the researcher's identification scheme.” Oh, I'm not sure there's a question in there. If a grant recipient is going through NIJ, this requirement's been in place for 15 years, something like that.
ERIN PARTIN: If I may jump in, NIJ requires the submission of a data archiving plan on projects where there are partnerships with other entities and data use agreement, so data use or data sharing agreements must acknowledge archiving. If an NIJ grantee were to come and request data, it would be in their data use agreement that the data would be archived. But again, any of these changes would lead to probably some site variations and language moving forward for the publication-based archiving.
Currently, we just acknowledge archiving at the end of the project period.
DR. ELIZABETH GROFF: Thank you, Erin. “So, PubMed is for manuscript. How does that work with publishing? Most journals have a ‘this will not be published elsewhere’ clause.”
DR. NANCY LA VIGNE: Well, my understanding is that's why we're encouraging the use of PubMed. So, we can get the manuscripts out there and not worry about the paywall issue or for our grantees to have to foot the bill for that. If they, for some reason, they want to foot the bill, it is an allowable component of the budget at the point of grant application to include resources for that purpose.
DR. GREGORY DUTTON: That is a good point. Generally, the author retains rights to their accepted manuscript. So, this is a process that NIH has been using since probably at least 2008, where their authors submit their author accepted manuscript. And the publishers have accepted this process for quite a while. This is not anything new. So, there had been a 12-month embargo generally on federally funded research. And now the move is toward no embargo, but the author posting their author-accepted manuscript is something that's been done for quite a while, so that's nothing new. It's generally recognized that when you accept an award from the government for the research, you agree to let the federal government have the rights to reuse of that work. So that posting relies on that.
And then to the other point that Nancy made, this is another question I think is out there. The question is, do NIJ grants currently or will they in the future be able to use grant funds to cover article processing charges, APCs in open access journals? Yes, absolutely. We've actually been inviting that in our programs for a period of years. So, if you pay for open access, you can certainly do that. For many journals, certainly some of the forensic sciences, it's all set up in PubMed Central so that if you publish open access in one of those journals, you don't have to do a thing. The journal will know when you tell them that it was NIJ-funded and you give the award number, they take care of it all for you.
DR. ELIZABETH GROFF: To be clear, PubMed Central putting your author accepted manuscript. You're not publishing with PubMed. You're still publishing with a journal, but you're making it available through PubMed.
DR. NANCY LA VIGNE: It will be the same as using ResearchGate that some people use, but we just are recommending PubMed Central because it's--why? I don't know because we've assessed that it's probably the better channel for us.
DR. GREGORY DUTTON: It's efficiently run. It's kind of the central place to look. And I'd say that with the federal purpose license, the federal right to reuse the work is the key to the authors being able to post their own author-accepted manuscripts.
DR. NANCY LA VIGNE: So, there's some protection there.
DR. ELIZABETH GROFF: This questioner is not sure whether the question is beyond the scope of webinar, but I am going to ask it: “does DOJ have data available for researchers to perform secondary analysis?”
ERIN PARTIN: Yes. The answer to your question is a resounding “yes.” If you're not familiar with the National Archive of Criminal Justice Data, the NACJD, I would strongly encourage you to peruse their website.
You can search by topic, by timeframe. You can see all of the NIJ-funded research that has been archived there. You can find links to final publications. You can find the data that's available. If it's non-restricted data, you can download it right now. If it is restricted, you can follow their application process to request access to it. You can reach out to PIs. All of that is at your fingertips, and we strongly encourage folks to use these resources. NIJ funds this research. And if it can be used for secondary analyses, that's fantastic.
DR. ELIZABETH GROFF: Thank you, Erin. “Does NIJ plan to require any specific information about the data collection process or standards for data that is collected in studies specifically thinking about who collects data sources?”
Well, we already have our data archiving plan. Hmm.
DR. NANCY LA VIGNE: We do. I mean, I don't know what's in the metadata. It's been a while since I've archived an NIJ dataset. Do we include the source?
DR. ELIZABETH GROFF: Well, there's tons of information.
DR. NANCY LA VIGNE: Like agency source?
DR. ELIZABETH GROFF: The NACJD has literally pages and pages of information about how to adequately document your data. Feel free to put a comment in the Federal Register request about that too.
DR. NANCY LA VIGNE: There is the question here that I'm not entirely clear on, which is, “Who contributes to the authoring of NIJ grants?”
And I'm wondering, does the person posing the question mean to say grants or solicitations? Because the authors of the NIJ grants are the research applicants that write the proposals that once funded become grants. So, if you're out there and that doesn't answer your question, please clarify your question. Thank you.
DR. ELIZABETH GROFF: I've got one other one like that. “What is the impact for full DNA profiles for individuals?”
DR. GREGORY DUTTON: Correct me if I'm wrong, but I think in general with this plan, we're not proposing any changes to the way that NIJ data is handled and archived. It's just the timeframe. So, this would be an expectation towards an incremental data availability, but all those complex issues about what types of data is maybe sensitive and should not be fully published, we're not suggesting any changes to that. So, the same types of protocols that you go through to determine that still apply, right?
DR. ELIZABETH GROFF: Yeah. And with that great note, we seem to have run out of questions.
DR. NANCY LA VIGNE: And time.
DR. ELIZABETH GROFF: And time. Well, there you go.
DR. NANCY LA VIGNE: Look at that. Amazing work, team!
DR. ELIZABETH GROFF: Yes. And thank you. Please, whoever's left on here, please comment on our Federal Register. Yes, request for comments. We would really, really appreciate your input. Thanks for attending.
DR. NANCY LA VIGNE: Thanks, everyone. I want to thank all the NIJ team members for this and certainly those who participated. Your questions and input are what is going to help shape this policy to meet both your needs and the needs of NIJ and the field. So, thank you so much.
STACY LEE: This will end our presentation.