CodeProject.AI Discussions

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

13-May-24 9:00

Server version:   2.6.2
System:           Windows
Operating System: Windows (Microsoft Windows 10.0.19045)
CPUs:             Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz (Intel)
                  1 CPU x 4 cores. 4 logical processors (x64)
GPU (Primary):    Intel(R) HD Graphics 630 (1,024 MiB) (Intel Corporation) 
                  Driver: 31.0.101.2111
System RAM:       16 GiB
Platform:         Windows
BuildConfig:      Release
Execution Env:    Native
Runtime Env:      Production
Runtimes installed:
  .NET runtime:     7.0.5
  .NET SDK:         Not found
  Default Python:   Not found
  Go:               Not found
  NodeJS:           Not found
  Rust:             Not found
Video adapter info:
  Intel(R) HD Graphics 630:
    Driver Version     31.0.101.2111
    Video Processor    Intel(R) HD Graphics Family
System GPU info:
  GPU 3D Usage       0%
  GPU RAM Usage      0
Global Environment variables:
  CPAI_APPROOTPATH = <root>
  CPAI_PORT        = 32168

Sean Ewington15-May-24 1:32

Sean Ewington

15-May-24 1:32

Thanks very much for that. And are you able to confirm that the GPU is actually being used for that module, but on the CodeProject.AI Server dashboard, CPU is displayed?

Thanks,
Sean Ewington
CodeProject

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

Mr Dunu14-May-24 14:48

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

14-May-24 14:48

I'm experiencing the exact same issue:

Server version:   2.6.2
System:           Windows
Operating System: Windows (Microsoft Windows 10.0.19045)
CPUs:             Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz (Intel)
                  1 CPU x 4 cores. 8 logical processors (x64)
GPU (Primary):    Intel(R) Iris(R) Plus Graphics 655 (1,024 MiB) (Intel Corporation) 
                  Driver: 31.0.101.2127
System RAM:       16 GiB
Platform:         Windows
BuildConfig:      Release
Execution Env:    Native
Runtime Env:      Production
Runtimes installed:
  .NET runtime:     7.0.7
  .NET SDK:         Not found
  Default Python:   3.10.7
  Go:               Not found
  NodeJS:           12.16.3
  Rust:             Not found
Video adapter info:
  Intel(R) Iris(R) Plus Graphics 655:
    Driver Version     31.0.101.2127
    Video Processor    Intel(R) Iris(R) Plus Graphics Family
System GPU info:
  GPU 3D Usage       0%
  GPU RAM Usage      0
Global Environment variables:
  CPAI_APPROOTPATH = <root>
  CPAI_PORT        = 32168

Sean Ewington15-May-24 1:32

Sean Ewington

15-May-24 1:32

Thanks very much for that. And are you able to confirm that the GPU is actually being used for that module, but on the CodeProject.AI Server dashboard, CPU is displayed?

Thanks,
Sean Ewington
CodeProject

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

Steve Winn15-May-24 1:48

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

15-May-24 1:48

I haven't been back to the machine yet, but the system info screen shows GPU usage=0.
Looked at the performance monitor just now, that shows occasional 3d activity, but nothing else. 1% memory constant. GPU0
Maybe it's not using the GPU.
I again clicked enable GPU, log says GPU enabled when the module restarts.
Maybe uninstall reinstall the module? I've uninstalled and reinstalled ver. 2.6.2 a couple of times already trying to figure out the Coral failed inferences. I just don't know.

Mr Dunu15-May-24 2:01

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

15-May-24 2:01

In my case, GPU is not used.

CPU (DirectML) is utilised despite selecting Enable GPU

Chris Maunder15-May-24 2:38

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

15-May-24 2:38

This is a naming issue. Here's the comment for the EnableGPU property:

/// Gets or sets a value indicating whether this process should enable GPU functionality
/// when running. This doesn't direct that a GPU must be used, but instead alerts that app
/// that it should enable GPUs if possible. Setting this to false means "even if you can
/// use a GPU, don't". Great for working around GPU issues that would sink the ship.
public bool? EnableGPU { get; set; } = true;

cheers
Chris Maunder

Mr Dunu15-May-24 4:22

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

15-May-24 4:22

How can we force the use of GPU?

Chris Maunder15-May-24 4:58

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

15-May-24 4:58

You can't. That module uses DirectML which means it will use the GPU if it can, meaning if the correct drivers are installed, and if the GPU is supported by DirectML. With DirectML, the use of the internal GPU is assumed, so if it's not actually using the GPU it's not something (AFAIK) we can really influence.

The only thing I would say is to be careful when trying to determine if the GPU is being used or not. A quote from an issue regarding DirectML using Tensorflow:

Quote:
One thing to keep in mind is that if you're using Task Manager to monitor GPU usage, it can sometimes be misleading because the default Task Manager GPU usage graph looks for 3D workloads which are different from compute workloads like tensorflow-directml

cheers
Chris Maunder

Mr Dunu15-May-24 5:04

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

15-May-24 5:04

Interesting. I do recall that prior to upgrading to CPAI ver2.6.2 (using 2.5.6) the YoloV5.NET model displayed that it was utilising the GPU, however, it seems to utilise CPU with this upgrade. I haven’t rolled back to CPAI ver2.5.6 to confirm my recollection though.

Steve Winn15-May-24 5:19

Re: CPAI ver 2.6.2 YOLOv5.NET CPU vs GPU on status screen?

15-May-24 5:19

Yes.
I'm going to try rolling back to 2.5.4.

Steve Winn2hrs 55mins ago

unable to install modules

2hrs 55mins ago

Me too.
Rolled back to CPAI ver. 2.5.6. YOLOv5 .NET connected to GPU on startup, and shows it is using GPU (Intel 630) without any user intervention.

Server version:   2.5.6
System:           Windows
Operating System: Windows (Microsoft Windows 10.0.19045)
CPUs:             Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz (Intel)
                  1 CPU x 4 cores. 4 logical processors (x64)
GPU (Primary):    Intel(R) HD Graphics 630 (1,024 MiB) (Intel Corporation) 
                  Driver: 31.0.101.2111
System RAM:       16 GiB
Platform:         Windows
BuildConfig:      Release
Execution Env:    Native
Runtime Env:      Production
Runtimes installed:
  .NET runtime:     7.0.5
  .NET SDK:         Not found
  Default Python:   Not found
  Go:               Not found
  NodeJS:           Not found
Video adapter info:
  Intel(R) HD Graphics 630:
    Driver Version     31.0.101.2111
    Video Processor    Intel(R) HD Graphics Family
System GPU info:
  GPU 3D Usage       10%
  GPU RAM Usage      0
Global Environment variables:
  CPAI_APPROOTPATH = <root>
  CPAI_PORT        = 32168

Module 'Object Detection (YOLOv5 .NET)' 1.9.3 (ID: ObjectDetectionYOLOv5Net)
Valid:         True
Module Path:   <root>\modules\ObjectDetectionYOLOv5Net
AutoStart:     True
Queue:         objectdetection_queue
Runtime:       dotnet
Runtime Loc:   Shared
FilePath:      bin\ObjectDetectionYOLOv5Net.exe
Pre installed: False
Start pause:   1 sec
Parallelism:   0
LogVerbosity:  
Platforms:     all
GPU Libraries: installed if available
GPU Enabled:   enabled
Accelerator:   
Half Precis.:  enable
Environment Variables
   CUSTOM_MODELS_DIR = <root>\modules\ObjectDetectionYOLOv5Net\custom-models
   MODELS_DIR        = <root>\modules\ObjectDetectionYOLOv5Net\assets
   MODEL_SIZE        = MEDIUM
Status Data:  {
  "inferenceDevice": "GPU",
  "inferenceLibrary": "DirectML",
  "canUseGPU": true,
  "successfulInferences": 179,
  "failedInferences": 0,
  "numInferences": 179,
  "averageInferenceMs": 245,
  "histogram": {
    "person": 63,
    "car": 262,
    "bus": 19,
    "truck": 2
  },
  "numItemsFound": 346
}
Started:      16 May 2024 7:52:55 AM Central Standard Time
LastSeen:     16 May 2024 8:07:00 AM Central Standard Time
Status:       Started
Requests:     179 (includes status calls)

todaxtream10-May-24 12:26

todaxtream

10-May-24 12:26

i have a fresh install of code project 2.6.4 on ubuntu server 24.04, object detection installed fine on on the install but trying to install LPR and Training and i get a 404 error for both of them

Server version:   2.6.4
System:           Linux
Operating System: Linux (Ubuntu 24.04)
CPUs:             12th Gen Intel(R) Core(TM) i7-12700K (Intel)
                  1 CPU x 12 cores. 20 logical processors (x64)
GPU (Primary):    NVIDIA GeForce RTX 2060 (6 GiB) (NVIDIA) 
                  Driver: 535.161.08, CUDA: 12.2 (up to: 12.2), Compute: 7.5, cuDNN: 
System RAM:       31 GiB
Platform:         Linux
BuildConfig:      Release
Execution Env:    Native
Runtime Env:      Production
Runtimes installed:
  .NET runtime:     7.0.18
  .NET SDK:         7.0.118
  Default Python:   3.12.3
  Go:               Not found
  NodeJS:           Not found
  Rust:             Not found
Video adapter info:
   TU104 [GeForce RTX 2060] (rev a1):
    Driver Version     
    Video Processor
System GPU info:
  GPU 3D Usage       10%
  GPU RAM Usage      1.9 GiB
Global Environment variables:
  CPAI_APPROOTPATH = <root>
  CPAI_PORT        = 32168

Re: unable to install modules

Chris Maunder15-May-24 5:07

2.6.2 AI 500 Errors with Blue Iris

15-May-24 5:07

Are you still getting 404s?

cheers
Chris Maunder

grid88-May-24 7:59

grid8

8-May-24 7:59

Upgraded to 2.6.2, Blue Iris intermittent AI Error 500 are back.

Can anyone explain the actual root cause of these errors? I'm getting a bit tired of the game between BI and CodeProject where updating one or the other has a 75% chance of re-introducing these errors.

Thanks

Chris Maunder8-May-24 8:15

8-May-24 8:15

I'm sure you've reported stuff before but it's extremely difficult for us to remember everyone's details on everything.

Can you please let us know

what version of the server you're using
on which module you're seeing the error
The actual error (screen shot would be good)
Have you tested the image that's throwing the 500 using the CodeProject.AI Explorer. It uses the same API as Blue Iris

cheers
Chris Maunder

09-05-24-1817 hosted at ImgBB — ImgBB[^]

grid89-May-24 4:19

grid8

9-May-24 4:19

For sure. For reference I was on 2.5.4 w/ Blue Iris 5.8.8.12 and had no AI errors.

Updated to CodeProject 2.6.2. Intermittent errors started every 15-30min on random cameras.
Decided to upgrade BI to latest 5.9.0.5. Still exist.

I'm using Coral TPU, tried various models and sizes. Multi-TPU disabled.

I don't know what images it's failing on unfortunately; BI doesn't give me the optics into that.

mailseth29-May-24 17:53

mailseth2

9-May-24 17:53

Does it work better if multi-TPU is enabled?

Chris Maunder15-May-24 5:09

15-May-24 5:09

Coral on Windows is not very reliable I'm afraid. It works, but intermittent errors are not uncommon. I use the .NET Object Detection module on Windows since it uses DirectML and so will make the best use of available hardware.

cheers
Chris Maunder

grid815-May-24 7:05

grid8

15-May-24 7:05

What do you recommend for Coral? Docker? Linux? I can pivot if it's known to be more reliable.

mailseth223hrs 52mins ago

mailseth2

23hrs 52mins ago

I’ve been running Ubuntu 20.04 and it’s been rock solid.

Chris Maunder20hrs 35mins ago