Tuesday, September 27, 2022
HomeHealthCustomized Software Troubleshooting With Cisco IOx

Customized Software Troubleshooting With Cisco IOx

Cisco IOx gives you with a platform to host your microservices on the networks edge, near your knowledge originator. Microservices let you course of your knowledge, apply filtering, aggregation, densification, high quality checks and way more!

What to bear in mind?

Are Linux based mostly programs and Docker purposes outdated mates of yours?  In that case, you most likely already learn about some challenges of making Docker photos. With IOx, creating your individual customized software is a well-defined course of. There are some subsequent steps throughout which you’ll create an software docker picture, outline operational parameters, and what number of assets to offer to your software.

Take into account the next as a guidelines to bear in mind, when creating an IOx docker-based software picture. Perhaps a number of the following challenges sound acquainted to you.

Potential challenges when creating Docker photos

1. Your ultimate Docker picture grew to become too massive?

Keep away from utilizing overly massive docker-base photos. Attempt to use a minimal one! For those who can handle to make use of “scratch” or “alpine” the lifecycle-management of your docker picture throughout deployment and redeployment will trigger much less overhead, ready time, and allocate much less bandwidth and storage on the machine. Examine the next fast comparability of various base picture sizes. Be mindful: your software framework, libraries, binaries, instruments, and interpreters will likely be added on prime of this dimension.


Word: Good candidates are almost definitely “ubuntu” with ~73MB, and even higher “alpine” with solely ~6MB base picture dimension. In case your software has minimal exterior dependencies, the optimum selection could be to construct your software picture from the smallest doable base picture “scratch,” which is even smaller than alpine. The extra effort for utilizing smaller photos vs. extra handy and bigger base picture, is effectively invested time!

2. Did you run out of storage when deploying an software bundle?
Take into account that you can’t declare all of the out there storage to IOx to your software. What you’ll deploy as an software bundle is a compressed archive containing the docker filesystem layers. Your machine will already allocate storage to obtain, retailer, and uncompress your picture. That in flip implies that successfully your picture will trigger storage to be allotted roughly 2.5 instances of its personal dimension!

3. Is your software storing knowledge steadily?
If the applying you need to run on IOx is utilizing storage to persist info, please bear in mind to equip the machine with an SSD enlargement module to help your plan. That is to offer further capability and to keep away from the primary system flash from sporting out.


4. Does your software create massive container logfiles?
Take into account that stdout and stderr are being logged. It is best to solely log essential messages within the manufacturing model of your software utilizing the applying logfiles. You must also think about using a log rotation mechanism if the logs are getting massive.

5. Are you not sure about errors throughout the operation of your software?
Be certain your software is logging details about its operational wellbeing. The usual paths for logging are machine dependent. On IR8x9 you’ll use “/knowledge/logs” and on IR18xx and IR1101 you’ll use “/iox_data/logs” to retailer your individual logfiles.

To consult with the completely different areas with out hardcoding it, please verify the hyperlink beneath on environmental variables, which is able to summary the completely different paths for you!

Take into consideration time and size-based log rotation to forestall the filesystem from filling up with software logs. For those who follow the talked about log areas, it is possible for you to to entry the logfiles from any of the out there management planes – IoT Operations Dashboard, Native Supervisor, ioxclient, and FND. Your software log is clearly the go-to level should you expertise points throughout operation together with your software!


6. Your machine logfile protocols “not discovered” errors throughout startup?
Be sure to comply to the IOx default paths for software information and logs. To your software information they’re for example “/iox_data/appdata” for IR1101/IR1800 and “/knowledge/appdata” for IR8x9, whereas to your logs it’s “/iox_data/logs” and “/knowledge/logs”.

An instance message from machine logs:

{"log":"/bin/sh: /begin.sh: not foundn","stream":"stderr","time":"2022-04-28T14:01:12.945697208Z"} 

7. Is your machine operating out of assets whereas deploying a number of purposes?
In case you are not sure how a lot system assets to offer to your software to meet its activity with out overprovisioning it, you might make it run completely, and monitor its actual utilization of CPU and reminiscence contained in the container whereas being below reasonable operational load. You may then later regulate the assets provisioned to your software to match your remark.


8. Are you unable to entry ports that your software exposes?
For those who intend to show ports for communication, ensure that your machine configuration will use for example NAT and correct port publicity on the corresponding port of your Docker container occasion. Consider if assigning a set IPv4 to your container eases debugging an error scenario. Don’t forget to verify the port publicity in your Dockerfile, in addition to your bundle.yaml. As well as, the bundle.yaml will even outline the purposes entry to providers like GPS, accelerometer, serial port and extra.


9. Does your software refuse to begin on the goal {hardware}?
The structure varies between completely different Cisco IoT units. For instance, IR1101/IR18xx gives an ARM64 based mostly structure, whereas IR8x9 does present a x86_64 based mostly {hardware} structure. Be certain to have your base picture chosen and your software program compiled or cross-compiled for the right goal structure, or else it is going to fail to put in or begin up. Selecting the improper goal {hardware} structure to your machine will both lead to not with the ability to add/activate your picture, or the startup will fail. The latter will trigger a corresponding log entry in your machine logs to be written.


10. Are you not sure concerning the operational state of your software?
Leverage health-monitoring to your software! The built-in health-monitoring gives you with the likelihood to have your software steadily checked for proper operations. The results of the well being verify is routinely provisioned to the management aircraft of your selection.


11. Does your software fail to begin and there’s no software log created?
For those who can not discover your software log (see 8.), it could be on account of a misconfiguration of the machine itself. The go-to useful resource is the operational logging of your machine itself. Relying in your setup, you will see the logs by way of IoT Operations Dashboard, LocalManager, FND, or ioxclient.

12. Does your software begin and instantly cease once more?
Double verify in case your Dockerfile specifies a blocking command utilizing for instance CMD or The method you specify must run with out interruption, or your IOx software will cease. You would possibly have to catch exceptions should you use a script to begin processes.


13. Does your software begin, run for some time, and exit whereas being below load?
Examine your purposes reminiscence and file descriptor consumption. It’d leak on both of them. In case your software doesn’t correctly free reminiscence and/or filedescriptors, it is going to finally be forcefully ended by the kernel. Swapping reminiscence just isn’t an possibility! To verify, in case you are being impacted by this drawback, plot the consumption on reminiscence and filedescriptors over time.

The next instance illustrates the ever-increasing allocation on VSS/RES reminiscence of a course of. The dearth of saturation suggests a reminiscence leak, that finally will result in this course of being forcefully ended.

IOx 2

14. Does activating your IOx software fail with an “ID syntax error”?
Be sure to use your docker picture identify:tag throughout export, otherwise you would possibly expertise points with the applying naming in your Dockerfile, manifest, or bundle.yaml when activating it.

Instance from LocalManager:


15. Is your ultimate software picture not compliant throughout safety scans, and/or constructing time takes an excessive amount of time?
Attempt to harden your picture by eradicating all pointless binaries, libraries, and caches. This won’t solely cut back the footprint on storage however assist to cut back safety threads for software program packages and libraries you need to patch and monitor for updates. This additionally contains caches the system constructed throughout picture meeting. The best strategy is to leverage Docker multi-stage builds. You’d solely copy the required issues to the final construct stage and take away all the things pointless, like for example system binaries your manufacturing software picture doesn’t want. Multi-stage builds let you create debug photos and manufacturing photos on the similar time.


The above instance illustrates the distinction of “intermediate-stage” to “final-stage” relating to their dimension. The consequence docker picture is functionally equivalent in each circumstances. Each originate from completely different phases of the identical picture construct course of.


16. Are you aware if any of the parts you utilized in your software picture is being affected by safety points?
Take into account checking your Docker picture safety compliance with out there toolkits resembling “Docker Bench.”

17. Do you need to shortcut the testing of your software and solely deploy beforehand examined software photos?
To save lots of effort and time check your docker picture in your native surroundings earlier than creating an IOx software. Additionally contemplate ramping up a DevNet sandbox for testing and studying!


The place to go from right here?


Las Vegas
Be part of our each day livestream from the DevNet Zone throughout Cisco Dwell!

Keep Knowledgeable!
Join the DevNet Zone Cisco Dwell Electronic mail Information and be the primary to learn about particular periods and surprises whether or not you might be attending in individual or will have interaction with us on-line.

We’d love to listen to what you assume. Ask a query or depart a remark beneath.
And keep linked with Cisco DevNet on social!

LinkedIn | Twitter @CiscoDevNet | Fb | YouTube Channel




Please enter your comment!
Please enter your name here

Most Popular

Recent Comments