Weekend update on the slowness issue- Marginal gains on Friday.

We are still not seeing anything like a return to the original performance for the E-Filing Manager API.

Some changes were made by the State's vendor early Friday afternoon that appeared to result in a small improvement, but it's hard to tell for sure because Friday afternoon is a time when our load is usually lighter than the rest of the week.

We're hoping the small gain is still discernable on Monday. However, even if this gain is solid, we acknowledge that it was small and we are not even close to calling the issue resolved. There's still far to go.

Thursday Morning update on performance issues. Bottom line: No noticeable improvements.

Early this morning (Thursday, 7/28) Tyler Technologies took the State's E-Filing Manager (EFM) off-line to do some performance tuning. We have been watching closely this morning, and as of 9AM, it looks like no improvement resulted from this work. (sigh.)

We'll be reporting these results to Tyler Technologies shortly so they can get to work on next steps.

Note that at 9AM, our sample size is fairly small, so our conclusions may be adjusted as the day unfolds, but so far, we're calling this round of tuning ineffective.

E-filing will be offline at Midnight while Tyler Technologies works on performance tuning.

We have been notified that Indiana's E-Filing Manager (EFM) will be offline for roughly one hour starting at midnight tonight (7/27/22).

That's good to hear, because as you doubtless know, the recent migration of the EFM's servers to the cloud went very poorly, and we've all been struggling with an extremely slow system this week.

Given the results of previous tries at performance tuning, we're not going to do the happy-dance yet, but we're hopeful and will wait to see what the morning brings.

If you'd like more background on this saga, take a look at our previous three posts on this topic.

Update on slowness isues with the EFM as of 1PM Tuesday, 7/26

For what has transpired up to this point, take a look at our post from last night about what caused this issue.

Here's the new information from today:

Yesterday, the State's EFM was running so slowly that it was effectively unusable. We estimate that the EFM was taking 17 times as long as usual to respond to the EFSPs such as Doxpop.

Today, it started off looking good, but a little slow. However, as of Noon today, our estimate is that the EFM is still taking 7 times as log to respond as prior to the cloud upgrade.

We have passed this information along to Tyler Technologies along with an urgent request to essentially double whatever change they made last night. That change got them halfway to the goal, so we expect it should not be hard to travel the rest of the way.

We share your frustration with this extended problem. Please know that we're doing everything we can, and at this point we're waiting on the folks at Tyler to respond.

The E-Filing outage on Friday and Monday- What happened?

 To understand this explanation, you need to know that there are two distinct sections of the E-Filing system in Indiana:

1) The section you interact with as a filer is called an E-Filing Service Provider (EFSP). The largest EFSPs in Indiana are the Tyler system, the Indiana Prosecutors' CMS, The Public Defenders' CMS and Doxpop. We provide the part of the system you interact with to get your work done.

2) Behind the scenes, all of the EFSPs connect to a single E-Filing Manager (EFM). The EFM takes the filing information that the EFSPs assemble into envelopes and write that information to the Odyssey system used by the courts. Tyler Technologies operates Indiana's EFM.

So in our case, the primary flow of data is Doxpop > EFM > Courts.

On Thursday evening, Tyler migrated Indiana's EFM to a cloud-based server. It didn't go smoothly. The result was a nearly complete outage lasting from Thursday night through the close of business on Monday. During this time, the EFM was extremely slow- Slow enough that it was essentially unusable. 

On Friday, all EFSPs were affected. Over the weekend, Tyler managed to get the connection to their EFSP working well. However, the interface used by the Prosecutors, Public Defenders, Doxpop, and other smaller EFSPs was still unusable on Monday. Doxpop reported this early Monday morning, and provided Tyler with all of the information we had available.

At 9:50 PM on Monday, Tyler reported that they completely fixed the interface used by the EFSPs, including Doxpop, that were affected on Monday. We'll be monitoring the system closely on Tuesday morning to confirm this, and ask that our users quickly report any issues they run into so we can let Tyler know if the fix applied Monday evening did the job.

Thanks to all of our users for your extraordinary patience and kindness during Friday and Monday. We were having two very bad days, but you made it a lot better by being so understanding!

Similarly, although it's tempting to point fingers, we really can't complain about the team at Tyler. Upgrades of this magnitude are no fun for anyone. They were having a very bad day too, but Tyler stuck with it until all of the issues were resolved. We appreciate that their technicians were hard at work well into the evening tonight.

Did this outage cause you to miss any filing deadlines?  If so, please read our previous blog post, which explains how the trial rules address this situation.

Did the e-file outage on Friday through Monday cause you to miss a deadline?

On Friday, July 22nd, we noticed an increasing issue with slow or completely unresponsive response times being returned by the State's E-Filing Manager, which is the central system that routes E-Filing submissions to each court. 

Doxpop promptly reported the issue to the State's vendor for this system, and received confirmation they were "aware of an issue which is affecting filings in IN Prod and are currently investigating".

As of 5PM, this problem was still not resolved, and the State's vendor indicated that they would continue working on the problem into the weekend.

If these issues caused you to miss a deadline, please note that Trial Rule 87(J)(1)(c), essentially provides for a one-day extension when a lengthy e-filing system outage or issue prevents filers from meeting a deadline.

We expect that the Trial Courts will all be aware of the scope and timing of this outage via direct communication from Trial Court Technology. However, if the court you are working with requires an affidavit from Doxpop to attach as an exhibit to the motion described in TR 86(J)(1)(c), please let us know and we'll be happy to supply documentation in support of your motion. (Simply attaching a screen print of this blog post may be sufficient.)

On the appellate side, take a look at Rule 68(M)(1)(d), which appears to be nearly identical to the Trial Court rule on this topic.

USPS Postal Rate set to increase, how will this effect Delivery by Doxpop rates?

 

As announced by the USPS earlier this year, USPS postal rates are set to increase on July 10th, 2022. We've reviewed those changes and will be updating the pricing to our Delivery by Doxpop services in line with that pricing at the end of the day on July 8th due to the update happening over the weekend.

The rate updates are detailed but in short, the only increase you will see is the actual postage increase for services that utilize the USPS. The only exception is with First Class Mail and Certified Mail flat packages. With those, you will see the per-page printing fee slightly increase as that is how we account for additional per-ounce charges. 

You can review our full Delivery by Doxpop fees on our fee schedule here. 

If you have any questions or would like more details on these updates please don't hesitate to contact Doxpop Support at 866-369-7671 or support@doxpop.com.