Jul 22 18:48:44.883959 ip-10-0-136-84 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 22 18:48:44.883971 ip-10-0-136-84 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 22 18:48:44.883980 ip-10-0-136-84 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 22 18:48:44.884287 ip-10-0-136-84 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 22 18:48:55.055152 ip-10-0-136-84 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 22 18:48:55.055169 ip-10-0-136-84 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7b40cfc706e244c1b3f627e83dd9e11a --
Jul 22 18:50:49.818059 ip-10-0-136-84 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.194144    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200182    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200195    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200199    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200208    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200211    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200214    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200217    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:50:50.267475 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200220    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:50:50.259935 ip-10-0-136-84 systemd[1]: Started Kubernetes Kubelet.
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200223    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200226    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200229    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200232    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200235    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200238    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200241    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200244    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200246    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200250    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200255    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200258    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200261    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200263    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200266    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200269    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200272    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200274    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200277    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:50:50.268452 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200279    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200282    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200284    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200287    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200289    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200292    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200295    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200298    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200302    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200306    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200309    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200312    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200316    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200319    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200321    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200325    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200328    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200330    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200333    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:50:50.269120 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200336    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200338    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200341    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200344    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200348    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200351    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200355    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200358    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200360    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200363    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200365    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200368    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200371    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200373    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200376    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200378    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200381    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200384    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200386    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:50:50.270204 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200388    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200392    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200395    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200397    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200400    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200403    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200406    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200408    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200411    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200413    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200417    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200420    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200423    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200425    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200428    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200430    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200433    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200436    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200439    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200443    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:50:50.273549 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200446    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200448    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200451    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200453    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200456    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200459    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200462    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200464    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200466    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200469    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200472    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200474    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200477    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.200480    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201193    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201200    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201203    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201207    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201211    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201214    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:50:50.274694 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201216    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201219    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201222    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201225    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201228    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201231    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201233    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201236    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201239    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201241    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201245    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201251    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201253    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201256    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201259    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201262    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201265    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201268    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201270    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201273    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:50:50.282199 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201275    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201278    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201281    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201283    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201285    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201288    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201291    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201293    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201296    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201298    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201301    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201303    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201306    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201309    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201312    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201314    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201317    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201319    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201322    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201324    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:50:50.282870 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201327    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201329    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201332    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201338    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201342    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201345    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201347    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201351    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201354    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201358    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201361    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201364    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201367    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201369    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201372    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201375    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201378    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201381    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201384    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201386    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:50:50.283502 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201389    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201391    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201394    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201397    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201400    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201403    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201406    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201408    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201412    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201416    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201419    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201421    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201424    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201427    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201429    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201433    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201436    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201438    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201441    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:50:50.284332 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201443    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201446    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201450    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201454    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201457    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201460    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201463    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201465    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201467    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201470    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201473    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201475    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201478    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201480    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201883    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201889    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201894    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201897    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201900    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:50:50.285152 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201903    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201905    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201909    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201911    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201914    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201916    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201919    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201922    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201924    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201939    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201943    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201946    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201950    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201954    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201959    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201964    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201967    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201970    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:50:50.285783 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201972    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201975    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201977    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201980    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201983    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201985    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201988    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201990    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201993    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201996    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.201998    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202000    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202003    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202006    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202009    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202011    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202013    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202016    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202020    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202022    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:50:50.286413 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202025    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202027    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202029    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202033    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202036    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202039    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202042    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202046    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202049    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202052    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202055    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202058    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202060    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202063    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202065    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202068    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202070    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202073    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202075    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202078    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:50:50.287243 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202080    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202084    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202087    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202089    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202092    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202095    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202098    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202100    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202103    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202106    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202108    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202111    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202113    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202115    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202118    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202121    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202124    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202127    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202129    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202132    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:50:50.287872 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202134    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202137    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202139    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202142    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202144    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202147    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202149    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202152    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202154    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202157    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202159    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202162    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202164    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202173    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202175    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.202178    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202241    2558 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202247    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202253    2558 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202259    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202265    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 18:50:50.288740 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202269    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202275    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202281    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202286    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202291    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202294    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202299    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202302    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202306    2558 flags.go:64] FLAG: --cgroup-root=""
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202309    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202312    2558 flags.go:64] FLAG: --client-ca-file=""
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202315    2558 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202318    2558 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202322    2558 flags.go:64] FLAG: --cluster-domain=""
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202324    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202328    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202331    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202334    2558 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202338    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202341    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202344    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202347    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202350    2558 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202353    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 18:50:50.289576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202357    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202361    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202363    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202367    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202370    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202373    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202377    2558 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202380    2558 flags.go:64] FLAG: --enable-server="true"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202383    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202387    2558 flags.go:64] FLAG: --event-burst="100"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202390    2558 flags.go:64] FLAG: --event-qps="50"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202393    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202396    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202399    2558 flags.go:64] FLAG: --eviction-hard=""
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202405    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202408    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202411    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202414    2558 flags.go:64] FLAG: --eviction-soft=""
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202417    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202420    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202423    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202425    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202428    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202431    2558 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202434    2558 flags.go:64] FLAG: --feature-gates=""
Jul 22 18:50:50.290654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202438    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202441    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202444    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202447    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202450    2558 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202453    2558 flags.go:64] FLAG: --help="false"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202456    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202460    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202463    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202466    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202469    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202472    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202475    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202478    2558 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202481    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202485    2558 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202488    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202491    2558 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202494    2558 flags.go:64] FLAG: --kube-reserved=""
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202496    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202499    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202503    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202506    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202509    2558 flags.go:64] FLAG: --lock-file=""
Jul 22 18:50:50.291739 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202512    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202515    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202518    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202523    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202526    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202529    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202531    2558 flags.go:64] FLAG: --logging-format="text"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202534    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202538    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202540    2558 flags.go:64] FLAG: --manifest-url=""
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202543    2558 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202547    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202550    2558 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202554    2558 flags.go:64] FLAG: --max-pods="110"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202557    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202563    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202565    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202568    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202572    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202574    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202577    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202585    2558 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202588    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202592    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 18:50:50.292603 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202595    2558 flags.go:64] FLAG: --pod-cidr=""
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202598    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202603    2558 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202606    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202609    2558 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202612    2558 flags.go:64] FLAG: --port="10250"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202616    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202619    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02f409510ddb8a865"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202622    2558 flags.go:64] FLAG: --qos-reserved=""
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202625    2558 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202628    2558 flags.go:64] FLAG: --register-node="true"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202631    2558 flags.go:64] FLAG: --register-with-taints=""
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202635    2558 flags.go:64] FLAG: --registry-burst="10"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202638    2558 flags.go:64] FLAG: --registry-qps="5"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202640    2558 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202643    2558 flags.go:64] FLAG: --reserved-memory=""
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202647    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202650    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202653    2558 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202656    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202659    2558 flags.go:64] FLAG: --runonce="false"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202661    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202664    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202669    2558 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202671    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 18:50:50.294057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202675    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202678    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202680    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202684    2558 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202686    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202689    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202692    2558 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202729    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202792    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202799    2558 flags.go:64] FLAG: --system-cgroups=""
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202804    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202815    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202820    2558 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202827    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202842    2558 flags.go:64] FLAG: --tls-min-version=""
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202847    2558 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.202851    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.203592    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.203602    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.203608    2558 flags.go:64] FLAG: --v="2"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.203616    2558 flags.go:64] FLAG: --version="false"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.203623    2558 flags.go:64] FLAG: --vmodule=""
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.203635    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.203641    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204109    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:50:50.310187 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204116    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204120    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204123    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204128    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204132    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204138    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204141    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204144    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204147    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204150    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204153    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204156    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204159    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204162    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204164    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204167    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204170    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204173    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204177    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:50:50.312314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204180    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204185    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204188    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204191    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204194    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204197    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204200    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204203    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204206    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204209    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204212    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204214    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204217    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204220    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204223    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204226    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204228    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204230    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204234    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204237    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:50:50.314899 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204240    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204243    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204247    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204250    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204253    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204255    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204258    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204261    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204263    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204266    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204269    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204271    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204274    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204278    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204281    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204283    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204286    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204288    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204291    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204294    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:50:50.317137 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204297    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204299    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204302    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204304    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204307    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204309    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204313    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204315    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204318    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204321    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204324    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204327    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204329    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204332    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204334    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204337    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204339    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204342    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204344    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:50:50.318264 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204346    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204349    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204351    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204354    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204356    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204359    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204363    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204366    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204368    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204371    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204373    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204376    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204379    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204381    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204384    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204386    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204389    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204392    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204394    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:50:50.324228 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.204397    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.205056    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.211527    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.211543    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.211562    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.211568    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211623    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211628    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211631    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211635    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211638    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211641    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211644    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211647    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211649    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:50:50.325505 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211652    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211655    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211658    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211661    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211663    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211666    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211669    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211671    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211674    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211677    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211680    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211682    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211685    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211688    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211690    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211693    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211695    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211698    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211703    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:50:50.330016 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211706    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211710    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211713    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211716    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211718    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211721    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211724    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211727    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211729    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211732    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211735    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211737    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211740    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211743    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211759    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211763    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211766    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211768    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211771    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211774    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:50:50.330721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211776    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211779    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211781    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211784    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211787    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211790    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211792    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211795    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211797    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211800    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211805    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211808    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211812    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211815    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211822    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211824    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211828    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211830    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211833    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:50:50.331304 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211837    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211841    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211844    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211847    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211850    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211853    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211856    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211859    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211862    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211865    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211868    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211870    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211873    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211876    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211878    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211881    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211884    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211886    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211889    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:50:50.331946 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211891    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211894    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211896    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211899    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211902    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211904    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211906    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211910    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211915    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211917    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211920    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211922    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.211925    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.211930    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212036    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 18:50:50.332492 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212041    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212044    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212047    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212050    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212054    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212058    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212061    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212064    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212066    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212069    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212072    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212078    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212081    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212083    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212086    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212089    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212091    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212094    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212096    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 18:50:50.333044 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212100    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212104    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212107    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212109    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212112    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212115    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212120    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212123    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212126    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212129    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212131    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212134    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212136    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212139    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212141    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212144    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212146    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212149    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212151    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212154    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 18:50:50.431305 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212156    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212159    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212161    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212164    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212167    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212170    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212172    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212176    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212179    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212182    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212185    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212187    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212190    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212192    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212196    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212198    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212201    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212203    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212207    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212210    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 18:50:50.446630 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212213    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212215    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212218    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212221    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212223    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212225    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212228    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212230    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212233    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212235    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212238    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212240    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212243    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212245    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212247    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212250    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212253    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212256    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212259    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212262    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 18:50:50.447325 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212264    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212266    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212269    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212271    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212274    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212276    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212279    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212281    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212284    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212286    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212290    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212293    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212296    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212299    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212301    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212304    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212306    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212309    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 18:50:50.448171 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:50.212311    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.212316    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.212412    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.214455    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.215625    2558 server.go:1019] "Starting client certificate rotation"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.215728    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.215792    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.217716    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.218867    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.230113    2558 log.go:25] "Validated CRI v1 runtime API"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.233070    2558 log.go:25] "Validated CRI v1 image API"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.235381    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 18:50:50.449005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.238487    2558 fs.go:135] Filesystem UUIDs: map[42292141-c8d5-4e74-bd9a-9aaf13809165:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a343aafa-9fa2-4bf5-81b4-7befe56771e6:/dev/nvme0n1p4]
Jul 22 18:50:50.449422 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.238508    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 18:50:50.449623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.243523    2558 manager.go:216] Machine: {Timestamp:2026-07-22 18:50:50.242491539 +0000 UTC m=+0.274509061 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099253 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec274efa31d47e1b173eec8bfa820805 SystemUUID:ec274efa-31d4-7e1b-173e-ec8bfa820805 BootID:7b40cfc7-06e2-44c1-b3f6-27e83dd9e11a Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:25:3f:9e:37:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:25:3f:9e:37:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:9d:eb:f8:45:24 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 18:50:50.449623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.243624    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 18:50:50.449623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.243703    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 18:50:50.449623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.244623    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 18:50:50.449623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.244644    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-84.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 18:50:50.449623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.245291    2558 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 18:50:50.449623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.245300    2558 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 18:50:50.449623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.245308    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.245891    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.247167    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.248251    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.249901    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.250018    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.250043    2558 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.250052    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.250067    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.250076    2558 kubelet.go:398] "Adding apiserver pod source"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.250085    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.250433    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.250466    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.252403    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.253924    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.253951    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254530    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254547    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254554    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254559    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254568    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254573    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254581    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254587    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254593    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254599    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254607    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.254612    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.255869    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.255881    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.259003    2558 server.go:1268] "Started kubelet"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.259078    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.259273    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.259332    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.260518    2558 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.261336    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-84.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.261786    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.262166    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.263513    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 18:50:50.450033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.263729    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.267229    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-84.ec2.internal.18c4b14234e59513  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-84.ec2.internal,UID:ip-10-0-136-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-84.ec2.internal,},FirstTimestamp:2026-07-22 18:50:50.258978067 +0000 UTC m=+0.290995588,LastTimestamp:2026-07-22 18:50:50.258978067 +0000 UTC m=+0.290995588,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-84.ec2.internal,}"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.270040    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.270056    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.270182    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.270631    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.270634    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.270708    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.270810    2558 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.270819    2558 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.271045    2558 factory.go:153] Registering CRI-O factory
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.271059    2558 factory.go:223] Registration of the crio container factory successfully
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.271105    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.271115    2558 factory.go:55] Registering systemd factory
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.271123    2558 factory.go:223] Registration of the systemd container factory successfully
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.271141    2558 factory.go:103] Registering Raw factory
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.271154    2558 manager.go:1196] Started watching for new ooms in manager
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.271562    2558 manager.go:349] Starting recovery of all containers
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.272713    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.272813    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.280936    2558 manager.go:354] Recovery completed
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.285698    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.288168    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.288218    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.288235    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.288765    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.288776    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.288794    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.288796    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.289802    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.289828    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.289838    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.290111    2558 policy_none.go:49] "None policy: Start"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.290130    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 18:50:50.451287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.290143    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.291158    2558 policy_none.go:47] "Start"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.301101    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-84.ec2.internal.18c4b14236a370af  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-84.ec2.internal,UID:ip-10-0-136-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-84.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-84.ec2.internal,},FirstTimestamp:2026-07-22 18:50:50.288197807 +0000 UTC m=+0.320215330,LastTimestamp:2026-07-22 18:50:50.288197807 +0000 UTC m=+0.320215330,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-84.ec2.internal,}"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.311298    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-65kc6"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.311931    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-84.ec2.internal.18c4b14236a3ddb4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-84.ec2.internal,UID:ip-10-0-136-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-84.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-84.ec2.internal,},FirstTimestamp:2026-07-22 18:50:50.288225716 +0000 UTC m=+0.320243242,LastTimestamp:2026-07-22 18:50:50.288225716 +0000 UTC m=+0.320243242,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-84.ec2.internal,}"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.320148    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-65kc6"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.332325    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-84.ec2.internal.18c4b14236a41794  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-84.ec2.internal,UID:ip-10-0-136-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-84.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-84.ec2.internal,},FirstTimestamp:2026-07-22 18:50:50.288240532 +0000 UTC m=+0.320258054,LastTimestamp:2026-07-22 18:50:50.288240532 +0000 UTC m=+0.320258054,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-84.ec2.internal,}"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.336357    2558 manager.go:337] "Starting Device Plugin manager"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.336387    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.336397    2558 server.go:87] "Starting device plugin registration server"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.336621    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.336632    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.336964    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.337019    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.337025    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.337640    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.337691    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.436938    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.438082    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:50:50.452328 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.438112    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.438125    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.438156    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.443768    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.444955    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.444987    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.445011    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.445041    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.447482    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.449312    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.452945 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.449330    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-84.ec2.internal\": node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.545254    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal"]
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.545331    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.545422    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.547235    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.547266    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.547276    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.548419    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.548531    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.548559    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.549952    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.549982    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.549993    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.549952    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.550054    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.550068    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.551140    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.551163    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.551820    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.551840    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 18:50:50.553702 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.551856    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID"
Jul 22 18:50:50.636618 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.574351    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/84cbfcd8e120f068e999cf03e85b5712-config\") pod \"kube-apiserver-proxy-ip-10-0-136-84.ec2.internal\" (UID: \"84cbfcd8e120f068e999cf03e85b5712\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.636618 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.574374    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5e77c64262014c3934b51b92a11a12d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"5e77c64262014c3934b51b92a11a12d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.636618 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.574390    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e77c64262014c3934b51b92a11a12d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"5e77c64262014c3934b51b92a11a12d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.636618 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.591358    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-84.ec2.internal\" not found" node="ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.636618 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.596964    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-84.ec2.internal\" not found" node="ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.748635 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.646261    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:50.748635 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.675102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/84cbfcd8e120f068e999cf03e85b5712-config\") pod \"kube-apiserver-proxy-ip-10-0-136-84.ec2.internal\" (UID: \"84cbfcd8e120f068e999cf03e85b5712\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.748635 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.675127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5e77c64262014c3934b51b92a11a12d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"5e77c64262014c3934b51b92a11a12d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.748635 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.675143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e77c64262014c3934b51b92a11a12d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"5e77c64262014c3934b51b92a11a12d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.748635 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.675181    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e77c64262014c3934b51b92a11a12d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"5e77c64262014c3934b51b92a11a12d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.748635 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.675221    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/84cbfcd8e120f068e999cf03e85b5712-config\") pod \"kube-apiserver-proxy-ip-10-0-136-84.ec2.internal\" (UID: \"84cbfcd8e120f068e999cf03e85b5712\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.748635 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.675244    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5e77c64262014c3934b51b92a11a12d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"5e77c64262014c3934b51b92a11a12d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.748635 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.747353    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:50.856062 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.849876    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:50.979547 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.893367    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.979547 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:50.901355    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:50.979547 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:50.950063    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:51.095432 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:51.054087    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:51.205844 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:51.154315    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:51.286620 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.215011    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 18:50:51.286620 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.215179    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 18:50:51.286620 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:51.254867    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found"
Jul 22 18:50:51.286620 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.277637    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:50:51.396374 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.334100    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 18:45:50 +0000 UTC" deadline="2028-01-25 01:53:33.164296994 +0000 UTC"
Jul 22 18:50:51.396374 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.334132    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13231h2m41.830168514s"
Jul 22 18:50:51.396374 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.370900    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:51.396374 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.391177    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:50:51.396374 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.394935    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 18:50:51.396374 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.395831    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal"
Jul 22 18:50:51.497442 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.409182    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 18:50:51.662028 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:51.592557    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84cbfcd8e120f068e999cf03e85b5712.slice/crio-dd80213ed4d9fb5431fa99cb96a7ac21d333839cd915a03362c6a4c51028e3ad WatchSource:0}: Error finding container dd80213ed4d9fb5431fa99cb96a7ac21d333839cd915a03362c6a4c51028e3ad: Status 404 returned error can't find the container with id dd80213ed4d9fb5431fa99cb96a7ac21d333839cd915a03362c6a4c51028e3ad
Jul 22 18:50:51.662028 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:51.592742    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e77c64262014c3934b51b92a11a12d7.slice/crio-4af8b6938a2bcbb93cdcbfcd347e8dfadc0726a50ff2ac1f1000f5f925fd4c36 WatchSource:0}: Error finding container 4af8b6938a2bcbb93cdcbfcd347e8dfadc0726a50ff2ac1f1000f5f925fd4c36: Status 404 returned error can't find the container with id 4af8b6938a2bcbb93cdcbfcd347e8dfadc0726a50ff2ac1f1000f5f925fd4c36
Jul 22 18:50:51.662028 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.598471    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 18:50:51.773939 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:51.721444    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:50:52.095225 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.095138    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 18:50:52.251803 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.251705    2558 apiserver.go:52] "Watching apiserver"
Jul 22 18:50:52.260410 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.260379    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 18:50:52.261704 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.261672    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-l6nb4","openshift-multus/multus-sz6vt","openshift-multus/network-metrics-daemon-fg676","openshift-ovn-kubernetes/ovnkube-node-2ztqx","kube-system/global-pull-secret-syncer-78bmc","kube-system/konnectivity-agent-fq67r","openshift-cluster-node-tuning-operator/tuned-j5nsr","openshift-image-registry/node-ca-98jjr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal","openshift-multus/multus-additional-cni-plugins-s66xt","openshift-network-diagnostics/network-check-target-c6572","openshift-network-operator/iptables-alerter-h6gp5","kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"]
Jul 22 18:50:52.263931 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.263907    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:50:52.266930 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.266906    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.268270 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.268246    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 18:50:52.268367 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.268297    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 18:50:52.268803 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.268435    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cgln8\""
Jul 22 18:50:52.268803 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.268797    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:52.268966 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.268879    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:50:52.269944 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.269560    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 18:50:52.269944 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.269762    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 18:50:52.269944 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.269799    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 18:50:52.270148 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.270132    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jq4jg\""
Jul 22 18:50:52.271268 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.271246    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 18:50:52.271371 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.271302    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.273413 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.273377    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.274634 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.273760    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 18:50:52.274634 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.274085    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 18:50:52.274634 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.274239    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tcggk\""
Jul 22 18:50:52.274634 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.274454    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 18:50:52.277042 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.276015    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 18:50:52.277042 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.276552    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nrq5q\""
Jul 22 18:50:52.277042 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.276647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.277042 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.276769    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 18:50:52.278737 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.278270    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:52.278737 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.278408    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:50:52.280619 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.279318    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hjhpf\""
Jul 22 18:50:52.280619 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.279554    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 18:50:52.280619 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.279938    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 18:50:52.280962 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.280938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.285527 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.283530    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.285527 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.284550    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 18:50:52.285527 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.284705    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 18:50:52.285527 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.285208    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 18:50:52.285527 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.285236    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 18:50:52.292870 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.290916    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 18:50:52.292870 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.291633    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 18:50:52.292870 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.291898    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 18:50:52.292870 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.292264    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 18:50:52.292870 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.292469    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 18:50:52.292870 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.292679    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vqbvg\""
Jul 22 18:50:52.295853 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.293921    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 18:50:52.295853 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.294335    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296564    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296605    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-sys-fs\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296643    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2106254d-8297-442a-b586-3b8813e4ab8b-host\") pod \"node-ca-98jjr\" (UID: \"2106254d-8297-442a-b586-3b8813e4ab8b\") " pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296676    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-socket-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296709    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296738    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w48th\" (UniqueName: \"kubernetes.io/projected/5d66540b-78b6-4e93-ba13-1a3192e90837-kube-api-access-w48th\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296772    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-registration-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqtvs\" (UniqueName: \"kubernetes.io/projected/d21e4142-fe9f-48f6-951e-d4520913b26d-kube-api-access-fqtvs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296966    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7bjfx\""
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.297177    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.296866    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2106254d-8297-442a-b586-3b8813e4ab8b-serviceca\") pod \"node-ca-98jjr\" (UID: \"2106254d-8297-442a-b586-3b8813e4ab8b\") " pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.297863    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.297901    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b-agent-certs\") pod \"konnectivity-agent-fq67r\" (UID: \"3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b\") " pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.297942    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:50:52.298220 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.298020    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:50:52.299183 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.297937    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b-konnectivity-ca\") pod \"konnectivity-agent-fq67r\" (UID: \"3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b\") " pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:50:52.299183 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.298091    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-device-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.299183 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.298125    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfjzj\" (UniqueName: \"kubernetes.io/projected/2106254d-8297-442a-b586-3b8813e4ab8b-kube-api-access-xfjzj\") pod \"node-ca-98jjr\" (UID: \"2106254d-8297-442a-b586-3b8813e4ab8b\") " pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.302844 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.302824    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.306535 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.306157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.306535 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.306271    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bzgp6\""
Jul 22 18:50:52.306688 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.306607    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 18:50:52.307096 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.306919    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 18:50:52.310308 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.308637    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 18:50:52.310308 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.309001    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5mz6j\""
Jul 22 18:50:52.310308 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.309185    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 18:50:52.316141 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.315996    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g8fcf"
Jul 22 18:50:52.324647 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.324541    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g8fcf"
Jul 22 18:50:52.372993 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.372968    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 18:50:52.399461 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399337    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-daemon-config\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.399461 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399382    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-sysctl-d\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.399461 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399413    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-run\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.399723 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399483    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-host\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.399723 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399522    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5ce3f6a-854b-4e1a-85b0-8a860dead860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.399723 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399559    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.399723 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-conf-dir\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.399723 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-kubelet\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.399723 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399638    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.399723 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399648    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.399723 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399675    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-cni-dir\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.399723 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399702    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a6bbcded-2352-4a2f-ab3a-a6715fabcc1e-hosts-file\") pod \"node-resolver-l6nb4\" (UID: \"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e\") " pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399739    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2106254d-8297-442a-b586-3b8813e4ab8b-serviceca\") pod \"node-ca-98jjr\" (UID: \"2106254d-8297-442a-b586-3b8813e4ab8b\") " pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9aa887d6-195c-4777-aefb-6c11ff7411c5-host-slash\") pod \"iptables-alerter-h6gp5\" (UID: \"9aa887d6-195c-4777-aefb-6c11ff7411c5\") " pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399816    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-cnibin\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-sysctl-conf\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.399983    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5ce3f6a-854b-4e1a-85b0-8a860dead860-cni-binary-copy\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400012    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400037    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b-konnectivity-ca\") pod \"konnectivity-agent-fq67r\" (UID: \"3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b\") " pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400062    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-device-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400086    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-os-release\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400111    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-sys\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400135    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dede4d61-d740-4199-97e9-6fda370a2f9b-env-overrides\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400163    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwc4v\" (UniqueName: \"kubernetes.io/projected/dede4d61-d740-4199-97e9-6fda370a2f9b-kube-api-access-mwc4v\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400187    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jts7l\" (UniqueName: \"kubernetes.io/projected/c7215cb6-97bb-4eed-a999-7f0fee6010d8-kube-api-access-jts7l\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.400288 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400211    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-sysconfig\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400235    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-kubernetes\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400337    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-device-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400335    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2106254d-8297-442a-b586-3b8813e4ab8b-serviceca\") pod \"node-ca-98jjr\" (UID: \"2106254d-8297-442a-b586-3b8813e4ab8b\") " pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400362    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-lib-modules\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.400453    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400472    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400501    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-system-cni-dir\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.400516    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:50:52.900494413 +0000 UTC m=+2.932511935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-socket-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400594    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w48th\" (UniqueName: \"kubernetes.io/projected/5d66540b-78b6-4e93-ba13-1a3192e90837-kube-api-access-w48th\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400621    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p8wp\" (UniqueName: \"kubernetes.io/projected/3cef412a-162c-4fd7-8097-357929bac1ad-kube-api-access-5p8wp\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.400990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400647    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400685    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-systemd-units\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400711    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-cni-bin\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7svf2\" (UniqueName: \"kubernetes.io/projected/f5ce3f6a-854b-4e1a-85b0-8a860dead860-kube-api-access-7svf2\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-socket-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-hostroot\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400855    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6bbcded-2352-4a2f-ab3a-a6715fabcc1e-tmp-dir\") pod \"node-resolver-l6nb4\" (UID: \"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e\") " pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-slash\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b-konnectivity-ca\") pod \"konnectivity-agent-fq67r\" (UID: \"3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b\") " pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400912    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-run-ovn\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400936    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5ce3f6a-854b-4e1a-85b0-8a860dead860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.400963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b-agent-certs\") pod \"konnectivity-agent-fq67r\" (UID: \"3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b\") " pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401030    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-var-lib-kubelet\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401057    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-systemd\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.401590 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401088    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-kubelet-config\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401137    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dede4d61-d740-4199-97e9-6fda370a2f9b-ovnkube-config\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401167    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-cnibin\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401195    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-os-release\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401224    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2106254d-8297-442a-b586-3b8813e4ab8b-host\") pod \"node-ca-98jjr\" (UID: \"2106254d-8297-442a-b586-3b8813e4ab8b\") " pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401250    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jf52\" (UniqueName: \"kubernetes.io/projected/9aa887d6-195c-4777-aefb-6c11ff7411c5-kube-api-access-2jf52\") pod \"iptables-alerter-h6gp5\" (UID: \"9aa887d6-195c-4777-aefb-6c11ff7411c5\") " pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401275    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-var-lib-kubelet\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-run-openvswitch\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401326    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-node-log\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401299    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2106254d-8297-442a-b586-3b8813e4ab8b-host\") pod \"node-ca-98jjr\" (UID: \"2106254d-8297-442a-b586-3b8813e4ab8b\") " pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401360    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9aa887d6-195c-4777-aefb-6c11ff7411c5-iptables-alerter-script\") pod \"iptables-alerter-h6gp5\" (UID: \"9aa887d6-195c-4777-aefb-6c11ff7411c5\") " pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401391    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-var-lib-cni-bin\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401422    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-run-systemd\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401474    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 18:50:52.402251 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401484    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-registration-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401525    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7215cb6-97bb-4eed-a999-7f0fee6010d8-cni-binary-copy\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401555    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-socket-dir-parent\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401579    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-run-k8s-cni-cncf-io\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401622    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-etc-kubernetes\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401630    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-registration-dir\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401649    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-log-socket\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401676    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-cni-netd\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401705    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfjzj\" (UniqueName: \"kubernetes.io/projected/2106254d-8297-442a-b586-3b8813e4ab8b-kube-api-access-xfjzj\") pod \"node-ca-98jjr\" (UID: \"2106254d-8297-442a-b586-3b8813e4ab8b\") " pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401776    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cef412a-162c-4fd7-8097-357929bac1ad-tmp\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-var-lib-openvswitch\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401832    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-etc-openvswitch\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401857    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dede4d61-d740-4199-97e9-6fda370a2f9b-ovnkube-script-lib\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401883    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:50:52.402763 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401912    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-sys-fs\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401937    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-system-cni-dir\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401962    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-var-lib-cni-multus\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.401986    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dede4d61-d740-4199-97e9-6fda370a2f9b-ovn-node-metrics-cert\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402063    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-run-netns\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402159    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-sys-fs\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402177    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d66540b-78b6-4e93-ba13-1a3192e90837-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402197    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-run-multus-certs\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402229    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6zxn\" (UniqueName: \"kubernetes.io/projected/a6bbcded-2352-4a2f-ab3a-a6715fabcc1e-kube-api-access-x6zxn\") pod \"node-resolver-l6nb4\" (UID: \"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e\") " pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402284    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-dbus\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402321    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-run-netns\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402361    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqtvs\" (UniqueName: \"kubernetes.io/projected/d21e4142-fe9f-48f6-951e-d4520913b26d-kube-api-access-fqtvs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402387    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-modprobe-d\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.403244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.402408    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cef412a-162c-4fd7-8097-357929bac1ad-etc-tuned\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.408209 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.408145    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b-agent-certs\") pod \"konnectivity-agent-fq67r\" (UID: \"3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b\") " pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:50:52.418958 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.418838    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfjzj\" (UniqueName: \"kubernetes.io/projected/2106254d-8297-442a-b586-3b8813e4ab8b-kube-api-access-xfjzj\") pod \"node-ca-98jjr\" (UID: \"2106254d-8297-442a-b586-3b8813e4ab8b\") " pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.419953 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.419852    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w48th\" (UniqueName: \"kubernetes.io/projected/5d66540b-78b6-4e93-ba13-1a3192e90837-kube-api-access-w48th\") pod \"aws-ebs-csi-driver-node-6mdzf\" (UID: \"5d66540b-78b6-4e93-ba13-1a3192e90837\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.420826 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.420707    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqtvs\" (UniqueName: \"kubernetes.io/projected/d21e4142-fe9f-48f6-951e-d4520913b26d-kube-api-access-fqtvs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:52.452045 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.451924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" event={"ID":"5e77c64262014c3934b51b92a11a12d7","Type":"ContainerStarted","Data":"4af8b6938a2bcbb93cdcbfcd347e8dfadc0726a50ff2ac1f1000f5f925fd4c36"}
Jul 22 18:50:52.463816 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.463107    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" event={"ID":"84cbfcd8e120f068e999cf03e85b5712","Type":"ContainerStarted","Data":"dd80213ed4d9fb5431fa99cb96a7ac21d333839cd915a03362c6a4c51028e3ad"}
Jul 22 18:50:52.503367 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-run-multus-certs\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.503550 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503375    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6zxn\" (UniqueName: \"kubernetes.io/projected/a6bbcded-2352-4a2f-ab3a-a6715fabcc1e-kube-api-access-x6zxn\") pod \"node-resolver-l6nb4\" (UID: \"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e\") " pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.503550 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503402    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-dbus\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:52.503550 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503433    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-run-netns\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.503550 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503461    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-modprobe-d\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.503550 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cef412a-162c-4fd7-8097-357929bac1ad-etc-tuned\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.503550 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503541    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-daemon-config\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.503892 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-sysctl-d\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.503892 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503584    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-run\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.503892 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503603    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-host\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.503892 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503599    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-run-multus-certs\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.503892 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503748    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-dbus\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:52.503892 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503775    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-run-netns\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.504204 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503898    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-run\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.504204 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.503949    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-host\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.504204 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504059    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-sysctl-d\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.504204 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504107    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-modprobe-d\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.504204 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504122    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5ce3f6a-854b-4e1a-85b0-8a860dead860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.504204 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504163    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-conf-dir\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.504204 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504193    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-kubelet\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504222    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504229    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-daemon-config\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504234    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-conf-dir\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-cni-dir\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504329    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-cni-dir\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504369    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a6bbcded-2352-4a2f-ab3a-a6715fabcc1e-hosts-file\") pod \"node-resolver-l6nb4\" (UID: \"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e\") " pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504380    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-kubelet\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9aa887d6-195c-4777-aefb-6c11ff7411c5-host-slash\") pod \"iptables-alerter-h6gp5\" (UID: \"9aa887d6-195c-4777-aefb-6c11ff7411c5\") " pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504430    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a6bbcded-2352-4a2f-ab3a-a6715fabcc1e-hosts-file\") pod \"node-resolver-l6nb4\" (UID: \"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e\") " pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504432    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-cnibin\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504475    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9aa887d6-195c-4777-aefb-6c11ff7411c5-host-slash\") pod \"iptables-alerter-h6gp5\" (UID: \"9aa887d6-195c-4777-aefb-6c11ff7411c5\") " pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504486    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-cnibin\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504485    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-sysctl-conf\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.504519 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504527    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5ce3f6a-854b-4e1a-85b0-8a860dead860-cni-binary-copy\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504569    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-os-release\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504594    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-sys\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504618    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dede4d61-d740-4199-97e9-6fda370a2f9b-env-overrides\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-os-release\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-sysctl-conf\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504646    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwc4v\" (UniqueName: \"kubernetes.io/projected/dede4d61-d740-4199-97e9-6fda370a2f9b-kube-api-access-mwc4v\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504692    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jts7l\" (UniqueName: \"kubernetes.io/projected/c7215cb6-97bb-4eed-a999-7f0fee6010d8-kube-api-access-jts7l\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504708    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-sys\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504719    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-sysconfig\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504729    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f5ce3f6a-854b-4e1a-85b0-8a860dead860-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504743    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-kubernetes\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504790    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-lib-modules\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504814    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504828    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-sysconfig\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504840    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504868    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-system-cni-dir\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504897    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8wp\" (UniqueName: \"kubernetes.io/projected/3cef412a-162c-4fd7-8097-357929bac1ad-kube-api-access-5p8wp\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504904    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-kubernetes\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504943    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-lib-modules\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-systemd-units\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-cni-bin\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505003    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7svf2\" (UniqueName: \"kubernetes.io/projected/f5ce3f6a-854b-4e1a-85b0-8a860dead860-kube-api-access-7svf2\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505013    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-run-ovn-kubernetes\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505053    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-systemd-units\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505074    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-cni-bin\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.504974    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.505510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505121    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5ce3f6a-854b-4e1a-85b0-8a860dead860-cni-binary-copy\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505168    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-hostroot\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.505191    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505196    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6bbcded-2352-4a2f-ab3a-a6715fabcc1e-tmp-dir\") pod \"node-resolver-l6nb4\" (UID: \"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e\") " pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505198    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-system-cni-dir\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505221    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-slash\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505248    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-hostroot\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.505260    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret podName:a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0 nodeName:}" failed. No retries permitted until 2026-07-22 18:50:53.005241274 +0000 UTC m=+3.037258792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret") pod "global-pull-secret-syncer-78bmc" (UID: "a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505278    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-run-ovn\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505291    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-slash\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505307    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5ce3f6a-854b-4e1a-85b0-8a860dead860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505331    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-run-ovn\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505335    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-var-lib-kubelet\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505357    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dede4d61-d740-4199-97e9-6fda370a2f9b-env-overrides\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505360    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-systemd\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.506307 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505402    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-etc-systemd\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505414    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-kubelet-config\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505444    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dede4d61-d740-4199-97e9-6fda370a2f9b-ovnkube-config\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505458    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-var-lib-kubelet\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505470    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-cnibin\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505493    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-os-release\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505507    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-kubelet-config\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505545    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-cnibin\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505555    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6bbcded-2352-4a2f-ab3a-a6715fabcc1e-tmp-dir\") pod \"node-resolver-l6nb4\" (UID: \"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e\") " pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jf52\" (UniqueName: \"kubernetes.io/projected/9aa887d6-195c-4777-aefb-6c11ff7411c5-kube-api-access-2jf52\") pod \"iptables-alerter-h6gp5\" (UID: \"9aa887d6-195c-4777-aefb-6c11ff7411c5\") " pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505603    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-var-lib-kubelet\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505629    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-run-openvswitch\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505655    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-node-log\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505680    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9aa887d6-195c-4777-aefb-6c11ff7411c5-iptables-alerter-script\") pod \"iptables-alerter-h6gp5\" (UID: \"9aa887d6-195c-4777-aefb-6c11ff7411c5\") " pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.507129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505704    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-var-lib-cni-bin\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505710    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5ce3f6a-854b-4e1a-85b0-8a860dead860-os-release\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-run-systemd\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505754    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-node-log\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505755    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7215cb6-97bb-4eed-a999-7f0fee6010d8-cni-binary-copy\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505840    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cef412a-162c-4fd7-8097-357929bac1ad-var-lib-kubelet\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505814    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-socket-dir-parent\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505885    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-run-k8s-cni-cncf-io\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505894    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-var-lib-cni-bin\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-etc-kubernetes\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505924    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-multus-socket-dir-parent\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-log-socket\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505965    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-run-systemd\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505885    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-run-openvswitch\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505979    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-cni-netd\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.507945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506010    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cef412a-162c-4fd7-8097-357929bac1ad-tmp\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506011    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-run-k8s-cni-cncf-io\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506035    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-var-lib-openvswitch\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506054    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-log-socket\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506063    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-etc-openvswitch\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506084    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dede4d61-d740-4199-97e9-6fda370a2f9b-ovnkube-config\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506093    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-etc-kubernetes\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506109    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dede4d61-d740-4199-97e9-6fda370a2f9b-ovnkube-script-lib\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506137    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506163    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-system-cni-dir\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506190    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-var-lib-cni-multus\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506210    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-var-lib-openvswitch\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506214    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dede4d61-d740-4199-97e9-6fda370a2f9b-ovn-node-metrics-cert\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506263    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-run-netns\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506289    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7215cb6-97bb-4eed-a999-7f0fee6010d8-cni-binary-copy\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.508766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506333    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-run-netns\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.509554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.505855    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f5ce3f6a-854b-4e1a-85b0-8a860dead860-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.509554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506369    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-system-cni-dir\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.509554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506392    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-etc-openvswitch\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.509554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506163    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dede4d61-d740-4199-97e9-6fda370a2f9b-host-cni-netd\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.509554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506487    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9aa887d6-195c-4777-aefb-6c11ff7411c5-iptables-alerter-script\") pod \"iptables-alerter-h6gp5\" (UID: \"9aa887d6-195c-4777-aefb-6c11ff7411c5\") " pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.509554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506566    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7215cb6-97bb-4eed-a999-7f0fee6010d8-host-var-lib-cni-multus\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.509554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.506890    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dede4d61-d740-4199-97e9-6fda370a2f9b-ovnkube-script-lib\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.509554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.507916    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cef412a-162c-4fd7-8097-357929bac1ad-etc-tuned\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.509554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.509267    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dede4d61-d740-4199-97e9-6fda370a2f9b-ovn-node-metrics-cert\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.511579 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.510677    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cef412a-162c-4fd7-8097-357929bac1ad-tmp\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.515421 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.515294    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:50:52.515421 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.515311    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:50:52.515421 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.515322    2558 projected.go:196] Error preparing data for projected volume kube-api-access-tjbph for pod openshift-network-diagnostics/network-check-target-c6572: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:50:52.515421 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.515426    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph podName:8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b nodeName:}" failed. No retries permitted until 2026-07-22 18:50:53.01540451 +0000 UTC m=+3.047422021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tjbph" (UniqueName: "kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph") pod "network-check-target-c6572" (UID: "8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:50:52.518291 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.518231    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jts7l\" (UniqueName: \"kubernetes.io/projected/c7215cb6-97bb-4eed-a999-7f0fee6010d8-kube-api-access-jts7l\") pod \"multus-sz6vt\" (UID: \"c7215cb6-97bb-4eed-a999-7f0fee6010d8\") " pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.518291 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.518268    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6zxn\" (UniqueName: \"kubernetes.io/projected/a6bbcded-2352-4a2f-ab3a-a6715fabcc1e-kube-api-access-x6zxn\") pod \"node-resolver-l6nb4\" (UID: \"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e\") " pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.518893 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.518851    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7svf2\" (UniqueName: \"kubernetes.io/projected/f5ce3f6a-854b-4e1a-85b0-8a860dead860-kube-api-access-7svf2\") pod \"multus-additional-cni-plugins-s66xt\" (UID: \"f5ce3f6a-854b-4e1a-85b0-8a860dead860\") " pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.520056 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.520018    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p8wp\" (UniqueName: \"kubernetes.io/projected/3cef412a-162c-4fd7-8097-357929bac1ad-kube-api-access-5p8wp\") pod \"tuned-j5nsr\" (UID: \"3cef412a-162c-4fd7-8097-357929bac1ad\") " pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.521838 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.521798    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jf52\" (UniqueName: \"kubernetes.io/projected/9aa887d6-195c-4777-aefb-6c11ff7411c5-kube-api-access-2jf52\") pod \"iptables-alerter-h6gp5\" (UID: \"9aa887d6-195c-4777-aefb-6c11ff7411c5\") " pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.522302 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.522248    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwc4v\" (UniqueName: \"kubernetes.io/projected/dede4d61-d740-4199-97e9-6fda370a2f9b-kube-api-access-mwc4v\") pod \"ovnkube-node-2ztqx\" (UID: \"dede4d61-d740-4199-97e9-6fda370a2f9b\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.581424 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.581390    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:50:52.597922 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.597693    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf"
Jul 22 18:50:52.611903 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.611828    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-98jjr"
Jul 22 18:50:52.619930 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.619908    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l6nb4"
Jul 22 18:50:52.629899 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.629874    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j5nsr"
Jul 22 18:50:52.640024 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.640001    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:50:52.649057 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.649036    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s66xt"
Jul 22 18:50:52.660969 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.660948    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h6gp5"
Jul 22 18:50:52.668033 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.668009    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sz6vt"
Jul 22 18:50:52.910085 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:52.910006    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:52.910241 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.910175    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:50:52.910298 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:52.910254    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:50:53.910238277 +0000 UTC m=+3.942255807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:50:53.011041 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:53.010999    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:53.011228 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.011176    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:50:53.011276 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.011247    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret podName:a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0 nodeName:}" failed. No retries permitted until 2026-07-22 18:50:54.011230352 +0000 UTC m=+4.043247866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret") pod "global-pull-secret-syncer-78bmc" (UID: "a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:50:53.111761 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:53.111708    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:50:53.111937 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.111862    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:50:53.111937 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.111890    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:50:53.111937 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.111903    2558 projected.go:196] Error preparing data for projected volume kube-api-access-tjbph for pod openshift-network-diagnostics/network-check-target-c6572: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:50:53.112099 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.111965    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph podName:8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b nodeName:}" failed. No retries permitted until 2026-07-22 18:50:54.111946276 +0000 UTC m=+4.143963858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tjbph" (UniqueName: "kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph") pod "network-check-target-c6572" (UID: "8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:50:53.328209 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:53.328109    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:45:52 +0000 UTC" deadline="2028-03-27 08:42:11.945452081 +0000 UTC"
Jul 22 18:50:53.328209 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:53.328160    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14725h51m18.617296984s"
Jul 22 18:50:53.445510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:53.445469    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:53.445706 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:53.445473    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:53.445706 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.445622    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:50:53.445851 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.445740    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:50:53.589349 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:53.589317    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cef412a_162c_4fd7_8097_357929bac1ad.slice/crio-b5be8cd164d4167e2fb234393ec775eb62d87a67c53e7f74ca9a2a5216dd280e WatchSource:0}: Error finding container b5be8cd164d4167e2fb234393ec775eb62d87a67c53e7f74ca9a2a5216dd280e: Status 404 returned error can't find the container with id b5be8cd164d4167e2fb234393ec775eb62d87a67c53e7f74ca9a2a5216dd280e
Jul 22 18:50:53.614552 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:53.614525    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d66540b_78b6_4e93_ba13_1a3192e90837.slice/crio-646290e4a12a2d15f83c3379e7f160ef1574ba7e1c2130659178c0a762b234ec WatchSource:0}: Error finding container 646290e4a12a2d15f83c3379e7f160ef1574ba7e1c2130659178c0a762b234ec: Status 404 returned error can't find the container with id 646290e4a12a2d15f83c3379e7f160ef1574ba7e1c2130659178c0a762b234ec
Jul 22 18:50:53.619399 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:53.619369    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddede4d61_d740_4199_97e9_6fda370a2f9b.slice/crio-4711e70b776460cc702d17aa0971f56d9dc761886de7b4ba76b86a4e401fafbc WatchSource:0}: Error finding container 4711e70b776460cc702d17aa0971f56d9dc761886de7b4ba76b86a4e401fafbc: Status 404 returned error can't find the container with id 4711e70b776460cc702d17aa0971f56d9dc761886de7b4ba76b86a4e401fafbc
Jul 22 18:50:53.620385 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:53.620300    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ce3f6a_854b_4e1a_85b0_8a860dead860.slice/crio-5d7278577fdcbc4944356c993776e0543719d3834639fee97e19694b69f2bcdb WatchSource:0}: Error finding container 5d7278577fdcbc4944356c993776e0543719d3834639fee97e19694b69f2bcdb: Status 404 returned error can't find the container with id 5d7278577fdcbc4944356c993776e0543719d3834639fee97e19694b69f2bcdb
Jul 22 18:50:53.621358 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:53.621337    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa887d6_195c_4777_aefb_6c11ff7411c5.slice/crio-f3580e2584236d595b3745332aa8ccb6471c1b5ecfbf2f5c7f09a3ee43b010dc WatchSource:0}: Error finding container f3580e2584236d595b3745332aa8ccb6471c1b5ecfbf2f5c7f09a3ee43b010dc: Status 404 returned error can't find the container with id f3580e2584236d595b3745332aa8ccb6471c1b5ecfbf2f5c7f09a3ee43b010dc
Jul 22 18:50:53.622927 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:53.622901    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7215cb6_97bb_4eed_a999_7f0fee6010d8.slice/crio-36d0da21f0a4af998738a912b6c37669574d26b6cbbddb7ce581ba8c0f1123b4 WatchSource:0}: Error finding container 36d0da21f0a4af998738a912b6c37669574d26b6cbbddb7ce581ba8c0f1123b4: Status 404 returned error can't find the container with id 36d0da21f0a4af998738a912b6c37669574d26b6cbbddb7ce581ba8c0f1123b4
Jul 22 18:50:53.624546 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:53.624199    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6bbcded_2352_4a2f_ab3a_a6715fabcc1e.slice/crio-e90ab17d5d25ec27d41cbb2e6b5fa5f99f0179d53a6b0c66cb17d7e93bac2bd8 WatchSource:0}: Error finding container e90ab17d5d25ec27d41cbb2e6b5fa5f99f0179d53a6b0c66cb17d7e93bac2bd8: Status 404 returned error can't find the container with id e90ab17d5d25ec27d41cbb2e6b5fa5f99f0179d53a6b0c66cb17d7e93bac2bd8
Jul 22 18:50:53.626721 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:53.626696    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd62269_edb1_4c5c_9b70_3c7ea48c6d8b.slice/crio-dda30f012d2b3d138d8d62af95cc0e91ef0eeb97f0a23930ec2b2e29d4fa0488 WatchSource:0}: Error finding container dda30f012d2b3d138d8d62af95cc0e91ef0eeb97f0a23930ec2b2e29d4fa0488: Status 404 returned error can't find the container with id dda30f012d2b3d138d8d62af95cc0e91ef0eeb97f0a23930ec2b2e29d4fa0488
Jul 22 18:50:53.628107 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:50:53.628070    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2106254d_8297_442a_b586_3b8813e4ab8b.slice/crio-87af166aebd57b0332c30cf370ae1c96706a3c2e484e4e336e48fda7525160e1 WatchSource:0}: Error finding container 87af166aebd57b0332c30cf370ae1c96706a3c2e484e4e336e48fda7525160e1: Status 404 returned error can't find the container with id 87af166aebd57b0332c30cf370ae1c96706a3c2e484e4e336e48fda7525160e1
Jul 22 18:50:53.919879 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:53.919657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:53.920034 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.919845    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:50:53.920034 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:53.920009    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:50:55.919992551 +0000 UTC m=+5.952010066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:50:54.020382 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.020338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:54.020581 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:54.020510    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:50:54.020581 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:54.020570    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret podName:a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0 nodeName:}" failed. No retries permitted until 2026-07-22 18:50:56.020552204 +0000 UTC m=+6.052569715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret") pod "global-pull-secret-syncer-78bmc" (UID: "a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:50:54.121517 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.121402    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:50:54.121677 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:54.121566    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:50:54.121677 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:54.121592    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:50:54.121677 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:54.121607    2558 projected.go:196] Error preparing data for projected volume kube-api-access-tjbph for pod openshift-network-diagnostics/network-check-target-c6572: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:50:54.121873 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:54.121679    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph podName:8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b nodeName:}" failed. No retries permitted until 2026-07-22 18:50:56.121658755 +0000 UTC m=+6.153676285 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tjbph" (UniqueName: "kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph") pod "network-check-target-c6572" (UID: "8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:50:54.329396 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.329294    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:45:52 +0000 UTC" deadline="2028-02-27 11:45:43.277347039 +0000 UTC"
Jul 22 18:50:54.329396 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.329359    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14032h54m48.947994785s"
Jul 22 18:50:54.451676 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.451032    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:50:54.451676 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:54.451163    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:50:54.486556 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.486519    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" event={"ID":"84cbfcd8e120f068e999cf03e85b5712","Type":"ContainerStarted","Data":"342c7b3f282873a398e348a042f14e23ea9b13b353c8fe728b491b94757f89e7"}
Jul 22 18:50:54.491128 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.491085    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-98jjr" event={"ID":"2106254d-8297-442a-b586-3b8813e4ab8b","Type":"ContainerStarted","Data":"87af166aebd57b0332c30cf370ae1c96706a3c2e484e4e336e48fda7525160e1"}
Jul 22 18:50:54.494053 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.494024    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fq67r" event={"ID":"3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b","Type":"ContainerStarted","Data":"dda30f012d2b3d138d8d62af95cc0e91ef0eeb97f0a23930ec2b2e29d4fa0488"}
Jul 22 18:50:54.496732 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.496704    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h6gp5" event={"ID":"9aa887d6-195c-4777-aefb-6c11ff7411c5","Type":"ContainerStarted","Data":"f3580e2584236d595b3745332aa8ccb6471c1b5ecfbf2f5c7f09a3ee43b010dc"}
Jul 22 18:50:54.499577 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.499545    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s66xt" event={"ID":"f5ce3f6a-854b-4e1a-85b0-8a860dead860","Type":"ContainerStarted","Data":"5d7278577fdcbc4944356c993776e0543719d3834639fee97e19694b69f2bcdb"}
Jul 22 18:50:54.501959 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.501898    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf" event={"ID":"5d66540b-78b6-4e93-ba13-1a3192e90837","Type":"ContainerStarted","Data":"646290e4a12a2d15f83c3379e7f160ef1574ba7e1c2130659178c0a762b234ec"}
Jul 22 18:50:54.509884 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.509856    2558 generic.go:356] "Generic (PLEG): container finished" podID="5e77c64262014c3934b51b92a11a12d7" containerID="70a451adc72410269a452cb04478b026be22d8f674b30f98f88453b00339a078" exitCode=0
Jul 22 18:50:54.509992 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.509942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" event={"ID":"5e77c64262014c3934b51b92a11a12d7","Type":"ContainerDied","Data":"70a451adc72410269a452cb04478b026be22d8f674b30f98f88453b00339a078"}
Jul 22 18:50:54.515226 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.515173    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l6nb4" event={"ID":"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e","Type":"ContainerStarted","Data":"e90ab17d5d25ec27d41cbb2e6b5fa5f99f0179d53a6b0c66cb17d7e93bac2bd8"}
Jul 22 18:50:54.517406 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.517380    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sz6vt" event={"ID":"c7215cb6-97bb-4eed-a999-7f0fee6010d8","Type":"ContainerStarted","Data":"36d0da21f0a4af998738a912b6c37669574d26b6cbbddb7ce581ba8c0f1123b4"}
Jul 22 18:50:54.522327 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.522302    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" event={"ID":"dede4d61-d740-4199-97e9-6fda370a2f9b","Type":"ContainerStarted","Data":"4711e70b776460cc702d17aa0971f56d9dc761886de7b4ba76b86a4e401fafbc"}
Jul 22 18:50:54.526404 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.526353    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" podStartSLOduration=3.5263343110000003 podStartE2EDuration="3.526334311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:50:54.509069953 +0000 UTC m=+4.541087489" watchObservedRunningTime="2026-07-22 18:50:54.526334311 +0000 UTC m=+4.558351849"
Jul 22 18:50:54.542716 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:54.542671    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j5nsr" event={"ID":"3cef412a-162c-4fd7-8097-357929bac1ad","Type":"ContainerStarted","Data":"b5be8cd164d4167e2fb234393ec775eb62d87a67c53e7f74ca9a2a5216dd280e"}
Jul 22 18:50:55.446381 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:55.445647    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:55.446381 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:55.445820    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:50:55.446381 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:55.446229    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:55.446381 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:55.446327    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:50:55.567138 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:55.566706    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" event={"ID":"5e77c64262014c3934b51b92a11a12d7","Type":"ContainerStarted","Data":"208c1749481b7be90b693a8eaefee19e466ea0deddc9d098b752e308eeb7f7d1"}
Jul 22 18:50:55.939286 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:55.939195    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:55.939476 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:55.939420    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:50:55.939538 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:55.939492    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:50:59.939470904 +0000 UTC m=+9.971488423 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:50:56.040780 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:56.040733    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:56.040987 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:56.040964    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:50:56.041072 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:56.041039    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret podName:a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:00.041019488 +0000 UTC m=+10.073037018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret") pod "global-pull-secret-syncer-78bmc" (UID: "a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:50:56.141968 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:56.141189    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:50:56.141968 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:56.141425    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:50:56.141968 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:56.141459    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:50:56.141968 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:56.141474    2558 projected.go:196] Error preparing data for projected volume kube-api-access-tjbph for pod openshift-network-diagnostics/network-check-target-c6572: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:50:56.141968 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:56.141547    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph podName:8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b nodeName:}" failed. No retries permitted until 2026-07-22 18:51:00.141526434 +0000 UTC m=+10.173543987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tjbph" (UniqueName: "kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph") pod "network-check-target-c6572" (UID: "8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:50:56.446342 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:56.446216    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:50:56.446342 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:56.446340    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:50:57.445604 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:57.445542    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:57.445789 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:57.445686    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:50:57.445789 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:57.445761    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:57.445926 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:57.445886    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:50:58.446404 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:58.445731    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:50:58.446404 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:58.445944    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:50:59.446428 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:59.445946    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:50:59.446428 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:59.446086    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:50:59.446428 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:59.445947    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:59.447055 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:59.446528    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:50:59.975973 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:50:59.975929    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:50:59.976166 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:59.976078    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:50:59.976166 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:50:59.976142    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:51:07.976123219 +0000 UTC m=+18.008140729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:51:00.076568 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:00.076479    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:00.076708 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:00.076650    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:51:00.076792 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:00.076726    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret podName:a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:08.076703149 +0000 UTC m=+18.108720657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret") pod "global-pull-secret-syncer-78bmc" (UID: "a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:51:00.177650 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:00.177610    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:00.177869 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:00.177850    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:51:00.177955 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:00.177881    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:51:00.177955 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:00.177895    2558 projected.go:196] Error preparing data for projected volume kube-api-access-tjbph for pod openshift-network-diagnostics/network-check-target-c6572: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:51:00.178049 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:00.177957    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph podName:8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b nodeName:}" failed. No retries permitted until 2026-07-22 18:51:08.177939562 +0000 UTC m=+18.209957093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tjbph" (UniqueName: "kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph") pod "network-check-target-c6572" (UID: "8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:51:00.446661 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:00.446592    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:00.447073 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:00.446704    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:01.445377 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:01.445331    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:01.445551 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:01.445451    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:01.445551 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:01.445512    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:01.445676 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:01.445629    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:02.446805 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:02.445611    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:02.446805 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:02.445764    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:03.446213 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:03.446169    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:03.446386 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:03.446175    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:03.446386 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:03.446299    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:03.446511 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:03.446388    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:04.445996 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:04.445916    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:04.446418 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:04.446038    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:05.445761 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:05.445724    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:05.445930 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:05.445727    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:05.445930 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:05.445858    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:05.446046 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:05.445977    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:06.445468 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:06.445441    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:06.445681 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:06.445566    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:07.446084 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:07.446040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:07.446084 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:07.446071    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:07.446587 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:07.446167    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:07.446587 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:07.446267    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:08.035634 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:08.035581    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:08.035895 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:08.035785    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:51:08.035895 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:08.035868    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:51:24.03584496 +0000 UTC m=+34.067862531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:51:08.136899 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:08.136865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:08.137060 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:08.137006    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:51:08.137103 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:08.137064    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret podName:a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:24.137048681 +0000 UTC m=+34.169066194 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret") pod "global-pull-secret-syncer-78bmc" (UID: "a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:51:08.237624 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:08.237554    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:08.237807 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:08.237740    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:51:08.237807 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:08.237775    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:51:08.237807 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:08.237786    2558 projected.go:196] Error preparing data for projected volume kube-api-access-tjbph for pod openshift-network-diagnostics/network-check-target-c6572: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:51:08.237971 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:08.237837    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph podName:8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b nodeName:}" failed. No retries permitted until 2026-07-22 18:51:24.237822403 +0000 UTC m=+34.269839911 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tjbph" (UniqueName: "kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph") pod "network-check-target-c6572" (UID: "8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:51:08.445561 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:08.445527    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:08.445778 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:08.445622    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:09.446073 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:09.446032    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:09.446521 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:09.446053    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:09.446521 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:09.446175    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:09.446521 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:09.446215    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:10.446636 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:10.446602    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:10.447065 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:10.446694    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:11.445340 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:11.445285    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:11.445528 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:11.445356    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:11.445528 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:11.445450    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:11.445702 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:11.445632    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:12.445990 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.445841    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:12.446889 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:12.446098    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:12.607697 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.607487    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-98jjr" event={"ID":"2106254d-8297-442a-b586-3b8813e4ab8b","Type":"ContainerStarted","Data":"5444b75db1aee88362044f970326ed7658814b63c2f4b092f293104cb5a8f15f"}
Jul 22 18:51:12.608845 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.608817    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fq67r" event={"ID":"3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b","Type":"ContainerStarted","Data":"62bbbfc88edd942aca3302d7240048f908ce591c54383ece5684aaeb54f3fa0b"}
Jul 22 18:51:12.610250 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.610229    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5ce3f6a-854b-4e1a-85b0-8a860dead860" containerID="1c24ba2db3cba6a5391e241622f5a28a20dcf42d2d0ac74faf5840b6b8d73855" exitCode=0
Jul 22 18:51:12.610319 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.610303    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s66xt" event={"ID":"f5ce3f6a-854b-4e1a-85b0-8a860dead860","Type":"ContainerDied","Data":"1c24ba2db3cba6a5391e241622f5a28a20dcf42d2d0ac74faf5840b6b8d73855"}
Jul 22 18:51:12.611805 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.611780    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf" event={"ID":"5d66540b-78b6-4e93-ba13-1a3192e90837","Type":"ContainerStarted","Data":"8afa74bf9be66669de87b98ad4f0fdde06cca15090787376f89f006cac21a671"}
Jul 22 18:51:12.613256 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.613231    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l6nb4" event={"ID":"a6bbcded-2352-4a2f-ab3a-a6715fabcc1e","Type":"ContainerStarted","Data":"5d3958ded4a8f1ef26a8fdd647cf3e2310b1f2baba5daf659c54adda6d2c6b2b"}
Jul 22 18:51:12.614725 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.614704    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sz6vt" event={"ID":"c7215cb6-97bb-4eed-a999-7f0fee6010d8","Type":"ContainerStarted","Data":"6954984bff9aed4608fcdbcdc5296e94b6280d8c81b6b405fe126e5e59d9b7dc"}
Jul 22 18:51:12.618482 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.617121    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" event={"ID":"dede4d61-d740-4199-97e9-6fda370a2f9b","Type":"ContainerStarted","Data":"94f55ea1951f5b9d2154c769d9360475a72b44e7dca2ace9104f8f87345fc04f"}
Jul 22 18:51:12.618482 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.617190    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" event={"ID":"dede4d61-d740-4199-97e9-6fda370a2f9b","Type":"ContainerStarted","Data":"976d5d80dd089ae33da9abb2c81fe8b004f56c965e4a774b9a206c1b454c1de4"}
Jul 22 18:51:12.618482 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.617236    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" event={"ID":"dede4d61-d740-4199-97e9-6fda370a2f9b","Type":"ContainerStarted","Data":"66956287fc296ccc9371fa7c6ce0fc0327260b6d9849271a2c3a8af3d163b8ce"}
Jul 22 18:51:12.620548 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.620514    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j5nsr" event={"ID":"3cef412a-162c-4fd7-8097-357929bac1ad","Type":"ContainerStarted","Data":"07c72c2edb01d884af76b97de0ccfb9f4b92943bc971b1eac9a56f1685de50f8"}
Jul 22 18:51:12.625681 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.623889    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-98jjr" podStartSLOduration=4.0670513249999996 podStartE2EDuration="22.62386949s" totalImagesPullingTime="18.556818165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:12.6232193 +0000 UTC m=+22.655236829" watchObservedRunningTime="2026-07-22 18:51:12.62386949 +0000 UTC m=+22.655887021"
Jul 22 18:51:12.625681 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.624053    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" podStartSLOduration=21.624043986 podStartE2EDuration="21.624043986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:50:55.585245086 +0000 UTC m=+5.617262632" watchObservedRunningTime="2026-07-22 18:51:12.624043986 +0000 UTC m=+22.656061510"
Jul 22 18:51:12.669427 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.669385    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sz6vt" podStartSLOduration=3.053848933 podStartE2EDuration="21.669365857s" totalImagesPullingTime="18.615516924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:12.669135163 +0000 UTC m=+22.701152693" watchObservedRunningTime="2026-07-22 18:51:12.669365857 +0000 UTC m=+22.701383387"
Jul 22 18:51:12.684435 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.684390    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l6nb4" podStartSLOduration=4.125315815 podStartE2EDuration="22.684371139s" totalImagesPullingTime="18.559055324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:12.68401441 +0000 UTC m=+22.716031940" watchObservedRunningTime="2026-07-22 18:51:12.684371139 +0000 UTC m=+22.716388668"
Jul 22 18:51:12.700512 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.700419    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j5nsr" podStartSLOduration=3.121778621 podStartE2EDuration="21.700404448s" totalImagesPullingTime="18.578625827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:12.700197192 +0000 UTC m=+22.732214722" watchObservedRunningTime="2026-07-22 18:51:12.700404448 +0000 UTC m=+22.732421972"
Jul 22 18:51:12.716155 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:12.716085    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fq67r" podStartSLOduration=8.012753414 podStartE2EDuration="22.716065803s" totalImagesPullingTime="14.703312389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:12.715451366 +0000 UTC m=+22.747468894" watchObservedRunningTime="2026-07-22 18:51:12.716065803 +0000 UTC m=+22.748083334"
Jul 22 18:51:13.445957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:13.445927    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:13.446129 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:13.446033    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:13.446129 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:13.446083    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:13.446466 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:13.446215    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:13.626430 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:13.626394    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" event={"ID":"dede4d61-d740-4199-97e9-6fda370a2f9b","Type":"ContainerStarted","Data":"dba9208b9a6f7d7a1f0552e06b117b8ae939ca15d7e03570de5ead915e3b2890"}
Jul 22 18:51:13.626556 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:13.626501    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" event={"ID":"dede4d61-d740-4199-97e9-6fda370a2f9b","Type":"ContainerStarted","Data":"7ba56c7beb1211751d0cbce6fb89bb306777633682900af985417535109a04d3"}
Jul 22 18:51:13.626619 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:13.626556    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" event={"ID":"dede4d61-d740-4199-97e9-6fda370a2f9b","Type":"ContainerStarted","Data":"a6b3e127abe97c72959f0a23f2fc3c3f32d59f60748d9cf09482bb6b8b303feb"}
Jul 22 18:51:13.696069 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:13.695976    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 18:51:14.355959 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:14.355845    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T18:51:13.69599531Z","UUID":"54fee1ae-06ec-49bf-94c2-2b186eaae180","Handler":null,"Name":"","Endpoint":""}
Jul 22 18:51:14.357467 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:14.357448    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 18:51:14.357611 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:14.357490    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 18:51:14.445876 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:14.445844    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:14.446064 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:14.445951    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:14.631764 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:14.631649    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf" event={"ID":"5d66540b-78b6-4e93-ba13-1a3192e90837","Type":"ContainerStarted","Data":"0209ece76bb4a2ae095b332ce49d5ead9b1e25916e5c27969dfdc917b57c7b63"}
Jul 22 18:51:14.633263 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:14.633238    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h6gp5" event={"ID":"9aa887d6-195c-4777-aefb-6c11ff7411c5","Type":"ContainerStarted","Data":"558ac508d2c5a2bcc0ab4085325bdcaecea5b211e1b769ec2de37deb1c34efdb"}
Jul 22 18:51:15.226274 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.226237    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:51:15.226849 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.226816    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:51:15.240881 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.240827    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h6gp5" podStartSLOduration=5.677334353 podStartE2EDuration="24.240812318s" totalImagesPullingTime="18.563477965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:14.647181374 +0000 UTC m=+24.679198908" watchObservedRunningTime="2026-07-22 18:51:15.240812318 +0000 UTC m=+25.272829848"
Jul 22 18:51:15.446142 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.446104    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:15.446322 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.446111    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:15.446322 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:15.446251    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:15.446410 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:15.446333    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:15.637992 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.637948    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf" event={"ID":"5d66540b-78b6-4e93-ba13-1a3192e90837","Type":"ContainerStarted","Data":"20cd8af3d59d4fa7cc62504b50d2c4531950bcbfa1ac689f2073a42c6a61aeea"}
Jul 22 18:51:15.641598 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.641567    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" event={"ID":"dede4d61-d740-4199-97e9-6fda370a2f9b","Type":"ContainerStarted","Data":"7cfc507721f690c9793e03483004395e554a8352bd7ce51ad0081d8df30cdce5"}
Jul 22 18:51:15.641927 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.641913    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:51:15.642381 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.642360    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fq67r"
Jul 22 18:51:15.654689 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:15.654638    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mdzf" podStartSLOduration=4.927116368 podStartE2EDuration="25.654622404s" totalImagesPullingTime="20.727506036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:15.654514763 +0000 UTC m=+25.686532295" watchObservedRunningTime="2026-07-22 18:51:15.654622404 +0000 UTC m=+25.686639935"
Jul 22 18:51:16.445271 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:16.445239    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:16.445454 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:16.445364    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:17.445415 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:17.445223    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:17.446209 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:17.445290    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:17.446209 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:17.445530    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:17.446209 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:17.445580    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:17.648244 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:17.648158    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5ce3f6a-854b-4e1a-85b0-8a860dead860" containerID="af14bfe72341f1e3167b70f6f1ddb2747dcbe9d85664401af462da9fcaa7e0e6" exitCode=0
Jul 22 18:51:17.648411 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:17.648252    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s66xt" event={"ID":"f5ce3f6a-854b-4e1a-85b0-8a860dead860","Type":"ContainerDied","Data":"af14bfe72341f1e3167b70f6f1ddb2747dcbe9d85664401af462da9fcaa7e0e6"}
Jul 22 18:51:17.652004 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:17.651980    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" event={"ID":"dede4d61-d740-4199-97e9-6fda370a2f9b","Type":"ContainerStarted","Data":"a0ab9be7b38051c426a53cc4bb824d22ae49f884a53339a8b821b48d720b13ba"}
Jul 22 18:51:17.703369 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:17.703315    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx" podStartSLOduration=8.079995176 podStartE2EDuration="26.70329877s" totalImagesPullingTime="18.623303594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:17.702764876 +0000 UTC m=+27.734782397" watchObservedRunningTime="2026-07-22 18:51:17.70329877 +0000 UTC m=+27.735316300"
Jul 22 18:51:18.445513 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:18.445478    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:18.445974 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:18.445600    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:18.655270 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:18.654941    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:51:18.655428 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:18.655277    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:51:18.655428 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:18.655323    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:51:18.670909 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:18.670876    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:51:18.671123 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:18.671101    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:51:19.214341 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:19.214251    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6572"]
Jul 22 18:51:19.214547 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:19.214408    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:19.214547 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:19.214509    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:19.219887 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:19.219846    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fg676"]
Jul 22 18:51:19.220037 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:19.219972    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:19.220104 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:19.220086    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:19.220513 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:19.220488    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78bmc"]
Jul 22 18:51:19.220628 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:19.220599    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:19.220714 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:19.220694    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:19.658605 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:19.658576    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5ce3f6a-854b-4e1a-85b0-8a860dead860" containerID="713e6f07d59a96271de2396d5039b8e8401b5f4c8e1e36e52dfba377fbd06f0e" exitCode=0
Jul 22 18:51:19.659125 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:19.658666    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s66xt" event={"ID":"f5ce3f6a-854b-4e1a-85b0-8a860dead860","Type":"ContainerDied","Data":"713e6f07d59a96271de2396d5039b8e8401b5f4c8e1e36e52dfba377fbd06f0e"}
Jul 22 18:51:20.449666 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:20.449638    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:20.449870 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:20.449713    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:20.449870 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:20.449742    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:20.449870 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:20.449792    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:20.449870 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:20.449801    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:20.450050 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:20.449905    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:21.665343 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:21.665311    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5ce3f6a-854b-4e1a-85b0-8a860dead860" containerID="c16a9101ddfcbbbbff80944054c76b775fbdc92b5d607124cce08eb43a70fb99" exitCode=0
Jul 22 18:51:21.665729 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:21.665373    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s66xt" event={"ID":"f5ce3f6a-854b-4e1a-85b0-8a860dead860","Type":"ContainerDied","Data":"c16a9101ddfcbbbbff80944054c76b775fbdc92b5d607124cce08eb43a70fb99"}
Jul 22 18:51:22.445645 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:22.445569    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:22.445861 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:22.445578    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:22.445861 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:22.445714    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:22.445861 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:22.445773    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:22.445861 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:22.445857    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:22.446054 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:22.445924    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:24.062107 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:24.062002    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:24.062475 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.062132    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:51:24.062475 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.062195    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:51:56.062177822 +0000 UTC m=+66.094195330 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 18:51:24.163053 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:24.163005    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:24.163214 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.163145    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 18:51:24.163254 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.163214    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret podName:a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:56.163198085 +0000 UTC m=+66.195215598 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret") pod "global-pull-secret-syncer-78bmc" (UID: "a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0") : object "kube-system"/"original-pull-secret" not registered
Jul 22 18:51:24.264160 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:24.264124    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:24.264346 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.264280    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 18:51:24.264346 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.264303    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 18:51:24.264346 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.264316    2558 projected.go:196] Error preparing data for projected volume kube-api-access-tjbph for pod openshift-network-diagnostics/network-check-target-c6572: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:51:24.264488 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.264376    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph podName:8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b nodeName:}" failed. No retries permitted until 2026-07-22 18:51:56.264360401 +0000 UTC m=+66.296377930 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tjbph" (UniqueName: "kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph") pod "network-check-target-c6572" (UID: "8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 18:51:24.445541 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:24.445504    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:24.445728 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:24.445504    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:24.445728 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.445637    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:51:24.445728 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:24.445504    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:24.445913 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.445732    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c6572" podUID="8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b"
Jul 22 18:51:24.445913 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:24.445837    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78bmc" podUID="a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0"
Jul 22 18:51:26.224308 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.224082    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeReady"
Jul 22 18:51:26.224650 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.224378    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 18:51:26.258396 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.258358    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"]
Jul 22 18:51:26.261342 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.261216    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:26.262179 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.262158    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b8d4d897-nz2g7"]
Jul 22 18:51:26.263891 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.263872    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 22 18:51:26.264075 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.264062    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-k994z\""
Jul 22 18:51:26.264341 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.264326    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 22 18:51:26.264589 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.264575    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.267098 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.267076    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d66st\""
Jul 22 18:51:26.267211 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.267134    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 18:51:26.267211 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.267143    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 18:51:26.267211 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.267137    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 18:51:26.274303 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.274279    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"]
Jul 22 18:51:26.276298 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.276276    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 18:51:26.277882 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.277864    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z2tqt"]
Jul 22 18:51:26.280138 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.280120    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.280998 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.280977    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6bpcc"]
Jul 22 18:51:26.282765 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.282726    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 18:51:26.282864 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.282772    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-phz5q\""
Jul 22 18:51:26.282864 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.282782    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 18:51:26.282864 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.282832    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6bpcc"
Jul 22 18:51:26.282864 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.282839    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b8d4d897-nz2g7"]
Jul 22 18:51:26.287954 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.287933    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 18:51:26.288080 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.288065    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wjx5r\""
Jul 22 18:51:26.288254 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.288237    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 18:51:26.288653 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.288641    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 18:51:26.293887 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.293870    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6bpcc"]
Jul 22 18:51:26.297775 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.297736    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z2tqt"]
Jul 22 18:51:26.378431 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378395    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-image-registry-private-configuration\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.378623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378446    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dfc22088-efc9-4e37-926b-a00989038020-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:26.378623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378493    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2c835e5f-9588-49c1-a80b-f47b83adb40f-ca-trust-extracted\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.378623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378529    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-installation-pull-secrets\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.378623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378553    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:26.378623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378569    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/956d7809-f242-41c4-a268-f692575d0b17-cert\") pod \"ingress-canary-6bpcc\" (UID: \"956d7809-f242-41c4-a268-f692575d0b17\") " pod="openshift-ingress-canary/ingress-canary-6bpcc"
Jul 22 18:51:26.378623 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378586    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-certificates\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.378918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378632    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.378918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378654    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfmb9\" (UniqueName: \"kubernetes.io/projected/956d7809-f242-41c4-a268-f692575d0b17-kube-api-access-sfmb9\") pod \"ingress-canary-6bpcc\" (UID: \"956d7809-f242-41c4-a268-f692575d0b17\") " pod="openshift-ingress-canary/ingress-canary-6bpcc"
Jul 22 18:51:26.378918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378722    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef7993f-e88f-4430-86be-7a8367cdc27c-config-volume\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.378918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378803    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.378918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378837    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef7993f-e88f-4430-86be-7a8367cdc27c-tmp-dir\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.378918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-bound-sa-token\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.378918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378872    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44x9v\" (UniqueName: \"kubernetes.io/projected/5ef7993f-e88f-4430-86be-7a8367cdc27c-kube-api-access-44x9v\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.378918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378891    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-trusted-ca\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.378918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.378905    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-855lh\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-kube-api-access-855lh\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.424992 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.424955    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fq67r_3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b/konnectivity-agent/0.log" limit=1024
Jul 22 18:51:26.445914 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.445897    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:26.445996 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.445983    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:26.446171 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.446130    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:26.448564 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.448545    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 18:51:26.448564 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.448546    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 18:51:26.448908 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.448892    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pz9g5\""
Jul 22 18:51:26.448977 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.448946    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vbtbq\""
Jul 22 18:51:26.448977 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.448956    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 18:51:26.448977 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.448967    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 18:51:26.480300 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480206    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-trusted-ca\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.480300 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480249    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-855lh\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-kube-api-access-855lh\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.480300 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480281    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-image-registry-private-configuration\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.480571 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dfc22088-efc9-4e37-926b-a00989038020-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:26.480571 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2c835e5f-9588-49c1-a80b-f47b83adb40f-ca-trust-extracted\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.480571 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480375    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-installation-pull-secrets\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.480571 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480401    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:26.480571 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480422    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/956d7809-f242-41c4-a268-f692575d0b17-cert\") pod \"ingress-canary-6bpcc\" (UID: \"956d7809-f242-41c4-a268-f692575d0b17\") " pod="openshift-ingress-canary/ingress-canary-6bpcc"
Jul 22 18:51:26.480571 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480447    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-certificates\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.480571 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480483    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.480907 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.480575    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 18:51:26.480907 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.480591    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8d4d897-nz2g7: secret "image-registry-tls" not found
Jul 22 18:51:26.480907 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.480643    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:51:26.480907 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.480657    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls podName:2c835e5f-9588-49c1-a80b-f47b83adb40f nodeName:}" failed. No retries permitted until 2026-07-22 18:51:26.980635488 +0000 UTC m=+37.012653011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls") pod "image-registry-b8d4d897-nz2g7" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f") : secret "image-registry-tls" not found
Jul 22 18:51:26.480907 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfmb9\" (UniqueName: \"kubernetes.io/projected/956d7809-f242-41c4-a268-f692575d0b17-kube-api-access-sfmb9\") pod \"ingress-canary-6bpcc\" (UID: \"956d7809-f242-41c4-a268-f692575d0b17\") " pod="openshift-ingress-canary/ingress-canary-6bpcc"
Jul 22 18:51:26.480907 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.480776    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:26.980726287 +0000 UTC m=+37.012743807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:51:26.480907 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480824    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef7993f-e88f-4430-86be-7a8367cdc27c-config-volume\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.480907 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.480856    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.481284 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.480934    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:51:26.481284 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.480970    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls podName:5ef7993f-e88f-4430-86be-7a8367cdc27c nodeName:}" failed. No retries permitted until 2026-07-22 18:51:26.980958701 +0000 UTC m=+37.012976210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls") pod "dns-default-z2tqt" (UID: "5ef7993f-e88f-4430-86be-7a8367cdc27c") : secret "dns-default-metrics-tls" not found
Jul 22 18:51:26.481284 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.481041    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef7993f-e88f-4430-86be-7a8367cdc27c-tmp-dir\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.481284 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.481069    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-bound-sa-token\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.481284 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.481104    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44x9v\" (UniqueName: \"kubernetes.io/projected/5ef7993f-e88f-4430-86be-7a8367cdc27c-kube-api-access-44x9v\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.481284 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.481144    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-certificates\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.481573 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.481422    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5ef7993f-e88f-4430-86be-7a8367cdc27c-config-volume\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.481573 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.481436    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-trusted-ca\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.481573 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.481442    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dfc22088-efc9-4e37-926b-a00989038020-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:26.481573 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.481424    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2c835e5f-9588-49c1-a80b-f47b83adb40f-ca-trust-extracted\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.481766 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.481627    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef7993f-e88f-4430-86be-7a8367cdc27c-tmp-dir\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.484760 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.484725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-installation-pull-secrets\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.484875 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.484741    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-image-registry-private-configuration\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.484875 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.484771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/956d7809-f242-41c4-a268-f692575d0b17-cert\") pod \"ingress-canary-6bpcc\" (UID: \"956d7809-f242-41c4-a268-f692575d0b17\") " pod="openshift-ingress-canary/ingress-canary-6bpcc"
Jul 22 18:51:26.490016 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.489989    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfmb9\" (UniqueName: \"kubernetes.io/projected/956d7809-f242-41c4-a268-f692575d0b17-kube-api-access-sfmb9\") pod \"ingress-canary-6bpcc\" (UID: \"956d7809-f242-41c4-a268-f692575d0b17\") " pod="openshift-ingress-canary/ingress-canary-6bpcc"
Jul 22 18:51:26.490163 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.490050    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-855lh\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-kube-api-access-855lh\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.492130 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.492110    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44x9v\" (UniqueName: \"kubernetes.io/projected/5ef7993f-e88f-4430-86be-7a8367cdc27c-kube-api-access-44x9v\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.492324 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.492303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-bound-sa-token\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.510096 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.508138    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fq67r_3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b/konnectivity-agent/0.log" limit=1024
Jul 22 18:51:26.595339 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.595298    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6bpcc"
Jul 22 18:51:26.682095 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.682051    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5ce3f6a-854b-4e1a-85b0-8a860dead860" containerID="8c486a15ea165dd4a6421de7c4aaa8a36fc967529a649e719a54b63d2bd0146d" exitCode=0
Jul 22 18:51:26.682265 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.682124    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s66xt" event={"ID":"f5ce3f6a-854b-4e1a-85b0-8a860dead860","Type":"ContainerDied","Data":"8c486a15ea165dd4a6421de7c4aaa8a36fc967529a649e719a54b63d2bd0146d"}
Jul 22 18:51:26.732714 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.732567    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6bpcc"]
Jul 22 18:51:26.736698 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:51:26.736671    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956d7809_f242_41c4_a268_f692575d0b17.slice/crio-8a5a730463ef3fb022d6552939269c037b8931fcb3ecea882135cd4685f9fcfa WatchSource:0}: Error finding container 8a5a730463ef3fb022d6552939269c037b8931fcb3ecea882135cd4685f9fcfa: Status 404 returned error can't find the container with id 8a5a730463ef3fb022d6552939269c037b8931fcb3ecea882135cd4685f9fcfa
Jul 22 18:51:26.985852 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.985714    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:26.985852 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.985780    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:26.985852 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:26.985823    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:26.986067 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.985866    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:51:26.986067 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.985903    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:51:26.986067 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.985931    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:27.985916035 +0000 UTC m=+38.017933548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:51:26.986067 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.985947    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls podName:5ef7993f-e88f-4430-86be-7a8367cdc27c nodeName:}" failed. No retries permitted until 2026-07-22 18:51:27.985940432 +0000 UTC m=+38.017957940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls") pod "dns-default-z2tqt" (UID: "5ef7993f-e88f-4430-86be-7a8367cdc27c") : secret "dns-default-metrics-tls" not found
Jul 22 18:51:26.986067 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.985870    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 18:51:26.986067 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.985958    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8d4d897-nz2g7: secret "image-registry-tls" not found
Jul 22 18:51:26.986067 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:26.985983    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls podName:2c835e5f-9588-49c1-a80b-f47b83adb40f nodeName:}" failed. No retries permitted until 2026-07-22 18:51:27.985977236 +0000 UTC m=+38.017994745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls") pod "image-registry-b8d4d897-nz2g7" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f") : secret "image-registry-tls" not found
Jul 22 18:51:27.686987 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:27.686945    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6bpcc" event={"ID":"956d7809-f242-41c4-a268-f692575d0b17","Type":"ContainerStarted","Data":"8a5a730463ef3fb022d6552939269c037b8931fcb3ecea882135cd4685f9fcfa"}
Jul 22 18:51:27.691063 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:27.691036    2558 generic.go:356] "Generic (PLEG): container finished" podID="f5ce3f6a-854b-4e1a-85b0-8a860dead860" containerID="d87335729d1747d4115e2e9285d9459a57beac63caa68e8128de6726475b57a5" exitCode=0
Jul 22 18:51:27.691208 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:27.691081    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s66xt" event={"ID":"f5ce3f6a-854b-4e1a-85b0-8a860dead860","Type":"ContainerDied","Data":"d87335729d1747d4115e2e9285d9459a57beac63caa68e8128de6726475b57a5"}
Jul 22 18:51:27.993370 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:27.993279    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:27.993370 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:27.993352    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:27.993620 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:27.993423    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:27.993620 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:27.993427    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 18:51:27.993620 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:27.993448    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8d4d897-nz2g7: secret "image-registry-tls" not found
Jul 22 18:51:27.993620 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:27.993509    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls podName:2c835e5f-9588-49c1-a80b-f47b83adb40f nodeName:}" failed. No retries permitted until 2026-07-22 18:51:29.993490007 +0000 UTC m=+40.025507518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls") pod "image-registry-b8d4d897-nz2g7" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f") : secret "image-registry-tls" not found
Jul 22 18:51:27.993620 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:27.993524    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:51:27.993620 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:27.993575    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls podName:5ef7993f-e88f-4430-86be-7a8367cdc27c nodeName:}" failed. No retries permitted until 2026-07-22 18:51:29.993561676 +0000 UTC m=+40.025579208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls") pod "dns-default-z2tqt" (UID: "5ef7993f-e88f-4430-86be-7a8367cdc27c") : secret "dns-default-metrics-tls" not found
Jul 22 18:51:27.993620 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:27.993524    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:51:27.993620 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:27.993616    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:29.993606923 +0000 UTC m=+40.025624432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:51:28.287048 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:28.286934    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fq67r_3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b/konnectivity-agent/0.log" limit=1024
Jul 22 18:51:28.387235 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:28.387196    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fq67r_3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b/konnectivity-agent/0.log" limit=1024
Jul 22 18:51:28.697364 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:28.697332    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s66xt" event={"ID":"f5ce3f6a-854b-4e1a-85b0-8a860dead860","Type":"ContainerStarted","Data":"d43c62bfb0292348317d10adbe6d951156480af62c733d56f5f0148d8829277a"}
Jul 22 18:51:28.724003 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:28.723949    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s66xt" podStartSLOduration=8.213011574 podStartE2EDuration="37.72393169s" totalImagesPullingTime="29.510920116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:28.722027194 +0000 UTC m=+38.754044725" watchObservedRunningTime="2026-07-22 18:51:28.72393169 +0000 UTC m=+38.755949220"
Jul 22 18:51:28.847172 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:28.847129    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fq67r_3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b/konnectivity-agent/0.log" limit=1024
Jul 22 18:51:28.910847 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:28.910810    2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fq67r_3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b/konnectivity-agent/0.log" limit=1024
Jul 22 18:51:29.700999 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:29.700956    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6bpcc" event={"ID":"956d7809-f242-41c4-a268-f692575d0b17","Type":"ContainerStarted","Data":"3d579c57148c2365c4392b185449eff390f63a7167f4c2a2c7a7bc2bf64025d1"}
Jul 22 18:51:29.716412 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:29.716358    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6bpcc" podStartSLOduration=1.216777195 podStartE2EDuration="3.716345534s" totalImagesPullingTime="2.499568339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:29.715856845 +0000 UTC m=+39.747874375" watchObservedRunningTime="2026-07-22 18:51:29.716345534 +0000 UTC m=+39.748363063"
Jul 22 18:51:30.007233 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:30.007140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:30.007233 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:30.007199    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:30.007417 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:30.007283    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:51:30.007417 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:30.007289    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 18:51:30.007417 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:30.007313    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8d4d897-nz2g7: secret "image-registry-tls" not found
Jul 22 18:51:30.007417 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:30.007344    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls podName:5ef7993f-e88f-4430-86be-7a8367cdc27c nodeName:}" failed. No retries permitted until 2026-07-22 18:51:34.007330924 +0000 UTC m=+44.039348432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls") pod "dns-default-z2tqt" (UID: "5ef7993f-e88f-4430-86be-7a8367cdc27c") : secret "dns-default-metrics-tls" not found
Jul 22 18:51:30.007417 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:30.007369    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls podName:2c835e5f-9588-49c1-a80b-f47b83adb40f nodeName:}" failed. No retries permitted until 2026-07-22 18:51:34.007356713 +0000 UTC m=+44.039374276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls") pod "image-registry-b8d4d897-nz2g7" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f") : secret "image-registry-tls" not found
Jul 22 18:51:30.007417 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:30.007405    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:30.007665 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:30.007480    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:51:30.007665 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:30.007515    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:34.007503467 +0000 UTC m=+44.039520991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:51:34.039105 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:34.039066    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:34.039489 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:34.039140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:34.039489 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:34.039179    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:34.039489 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:34.039230    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:51:34.039489 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:34.039289    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 18:51:34.039489 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:34.039304    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8d4d897-nz2g7: secret "image-registry-tls" not found
Jul 22 18:51:34.039489 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:34.039306    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:51:34.039489 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:34.039304    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls podName:5ef7993f-e88f-4430-86be-7a8367cdc27c nodeName:}" failed. No retries permitted until 2026-07-22 18:51:42.039287545 +0000 UTC m=+52.071305057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls") pod "dns-default-z2tqt" (UID: "5ef7993f-e88f-4430-86be-7a8367cdc27c") : secret "dns-default-metrics-tls" not found
Jul 22 18:51:34.039489 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:34.039361    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls podName:2c835e5f-9588-49c1-a80b-f47b83adb40f nodeName:}" failed. No retries permitted until 2026-07-22 18:51:42.0393463 +0000 UTC m=+52.071363824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls") pod "image-registry-b8d4d897-nz2g7" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f") : secret "image-registry-tls" not found
Jul 22 18:51:34.039489 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:34.039380    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:42.039373343 +0000 UTC m=+52.071390851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:51:42.101056 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:42.101015    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:42.101486 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:42.101065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:42.101486 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:42.101163    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 18:51:42.101486 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:42.101165    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:51:42.101486 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:42.101189    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:42.101486 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:42.101239    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:58.101223157 +0000 UTC m=+68.133240664 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:51:42.101486 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:42.101258    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:51:42.101486 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:42.101175    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8d4d897-nz2g7: secret "image-registry-tls" not found
Jul 22 18:51:42.101486 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:42.101298    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls podName:5ef7993f-e88f-4430-86be-7a8367cdc27c nodeName:}" failed. No retries permitted until 2026-07-22 18:51:58.101283213 +0000 UTC m=+68.133300738 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls") pod "dns-default-z2tqt" (UID: "5ef7993f-e88f-4430-86be-7a8367cdc27c") : secret "dns-default-metrics-tls" not found
Jul 22 18:51:42.101486 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:42.101323    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls podName:2c835e5f-9588-49c1-a80b-f47b83adb40f nodeName:}" failed. No retries permitted until 2026-07-22 18:51:58.101309287 +0000 UTC m=+68.133326822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls") pod "image-registry-b8d4d897-nz2g7" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f") : secret "image-registry-tls" not found
Jul 22 18:51:50.681278 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:50.681250    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ztqx"
Jul 22 18:51:56.124901 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.124835    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:51:56.127872 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.127851    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 18:51:56.135973 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:56.135951    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 18:51:56.136037 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:56.136017    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:53:00.136000252 +0000 UTC m=+130.168017760 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : secret "metrics-daemon-secret" not found
Jul 22 18:51:56.226256 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.226216    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:56.229180 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.229156    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 18:51:56.240911 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.240877    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0-original-pull-secret\") pod \"global-pull-secret-syncer-78bmc\" (UID: \"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0\") " pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:56.327005 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.326951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:56.329782 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.329744    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 18:51:56.340168 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.340147    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 18:51:56.350499 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.350473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjbph\" (UniqueName: \"kubernetes.io/projected/8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b-kube-api-access-tjbph\") pod \"network-check-target-c6572\" (UID: \"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b\") " pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:56.458980 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.458902    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vbtbq\""
Jul 22 18:51:56.466221 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.466197    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:51:56.466287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.466224    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78bmc"
Jul 22 18:51:56.600616 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.600578    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c6572"]
Jul 22 18:51:56.605400 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:51:56.605374    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ded3a80_a9cd_4ce4_b30b_d0a00c3ecb9b.slice/crio-7d0b0b2208b2d1fa40daa7875060dc538c3ab67b3c1ccd3316a5649140aba97e WatchSource:0}: Error finding container 7d0b0b2208b2d1fa40daa7875060dc538c3ab67b3c1ccd3316a5649140aba97e: Status 404 returned error can't find the container with id 7d0b0b2208b2d1fa40daa7875060dc538c3ab67b3c1ccd3316a5649140aba97e
Jul 22 18:51:56.619420 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.619389    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78bmc"]
Jul 22 18:51:56.622420 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:51:56.622393    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a1f9ec_d356_47cb_857b_ec77d9a5a7e0.slice/crio-cc3330c2b81fc7f393ce2d99b0cab9ebde4efd4664dd4b748afea4ecfd4b8f75 WatchSource:0}: Error finding container cc3330c2b81fc7f393ce2d99b0cab9ebde4efd4664dd4b748afea4ecfd4b8f75: Status 404 returned error can't find the container with id cc3330c2b81fc7f393ce2d99b0cab9ebde4efd4664dd4b748afea4ecfd4b8f75
Jul 22 18:51:56.761634 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.761539    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6572" event={"ID":"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b","Type":"ContainerStarted","Data":"7d0b0b2208b2d1fa40daa7875060dc538c3ab67b3c1ccd3316a5649140aba97e"}
Jul 22 18:51:56.762528 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:56.762507    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-78bmc" event={"ID":"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0","Type":"ContainerStarted","Data":"cc3330c2b81fc7f393ce2d99b0cab9ebde4efd4664dd4b748afea4ecfd4b8f75"}
Jul 22 18:51:58.141927 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:58.141871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:51:58.142414 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:58.141940    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:51:58.142414 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:58.142046    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:51:58.142414 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:58.142051    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 18:51:58.142414 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:58.142126    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:52:30.142105569 +0000 UTC m=+100.174123080 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:51:58.142414 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:58.142132    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8d4d897-nz2g7: secret "image-registry-tls" not found
Jul 22 18:51:58.142414 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:51:58.142151    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:51:58.142414 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:58.142199    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls podName:2c835e5f-9588-49c1-a80b-f47b83adb40f nodeName:}" failed. No retries permitted until 2026-07-22 18:52:30.142180828 +0000 UTC m=+100.174198340 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls") pod "image-registry-b8d4d897-nz2g7" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f") : secret "image-registry-tls" not found
Jul 22 18:51:58.142414 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:58.142255    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:51:58.142414 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:51:58.142299    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls podName:5ef7993f-e88f-4430-86be-7a8367cdc27c nodeName:}" failed. No retries permitted until 2026-07-22 18:52:30.142285776 +0000 UTC m=+100.174303287 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls") pod "dns-default-z2tqt" (UID: "5ef7993f-e88f-4430-86be-7a8367cdc27c") : secret "dns-default-metrics-tls" not found
Jul 22 18:52:01.777824 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:52:01.777708    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-78bmc" event={"ID":"a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0","Type":"ContainerStarted","Data":"49e3e1b63f69ee9fc3132b3a74c0baa7987e10d1e16f8459a29fb19131592c5e"}
Jul 22 18:52:01.779176 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:52:01.779155    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c6572" event={"ID":"8ded3a80-a9cd-4ce4-b30b-d0a00c3ecb9b","Type":"ContainerStarted","Data":"574054819af9e02b9da843adfefb7b6d10f9327c497d4efba51e236062ef63c5"}
Jul 22 18:52:01.779291 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:52:01.779280    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:52:01.797160 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:52:01.797106    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-78bmc" podStartSLOduration=65.960985953 podStartE2EDuration="1m10.797092125s" totalImagesPullingTime="4.836106172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:01.796484371 +0000 UTC m=+71.828501903" watchObservedRunningTime="2026-07-22 18:52:01.797092125 +0000 UTC m=+71.829109654"
Jul 22 18:52:01.814806 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:52:01.814739    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c6572" podStartSLOduration=65.974106681 podStartE2EDuration="1m10.814722057s" totalImagesPullingTime="4.840615376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:01.814258708 +0000 UTC m=+71.846276237" watchObservedRunningTime="2026-07-22 18:52:01.814722057 +0000 UTC m=+71.846739587"
Jul 22 18:52:30.186137 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:52:30.186090    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:52:30.186137 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:52:30.186148    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:52:30.186610 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:52:30.186236    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 18:52:30.186610 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:52:30.186239    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:52:30.186610 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:52:30.186276    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:52:30.186610 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:52:30.186318    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:53:34.186301332 +0000 UTC m=+164.218318843 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:52:30.186610 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:52:30.186250    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8d4d897-nz2g7: secret "image-registry-tls" not found
Jul 22 18:52:30.186610 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:52:30.186362    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:52:30.186610 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:52:30.186391    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls podName:2c835e5f-9588-49c1-a80b-f47b83adb40f nodeName:}" failed. No retries permitted until 2026-07-22 18:53:34.186379518 +0000 UTC m=+164.218397030 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls") pod "image-registry-b8d4d897-nz2g7" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f") : secret "image-registry-tls" not found
Jul 22 18:52:30.186610 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:52:30.186404    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls podName:5ef7993f-e88f-4430-86be-7a8367cdc27c nodeName:}" failed. No retries permitted until 2026-07-22 18:53:34.186398322 +0000 UTC m=+164.218415830 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls") pod "dns-default-z2tqt" (UID: "5ef7993f-e88f-4430-86be-7a8367cdc27c") : secret "dns-default-metrics-tls" not found
Jul 22 18:52:32.784516 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:52:32.784479    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c6572"
Jul 22 18:53:00.210227 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:53:00.210182    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:53:00.210743 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:00.210340    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 18:53:00.210743 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:00.210407    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:55:02.210392058 +0000 UTC m=+252.242409566 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : secret "metrics-daemon-secret" not found
Jul 22 18:53:29.273175 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:29.273136    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh" podUID="dfc22088-efc9-4e37-926b-a00989038020"
Jul 22 18:53:29.279287 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:29.279257    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7" podUID="2c835e5f-9588-49c1-a80b-f47b83adb40f"
Jul 22 18:53:29.290562 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:29.290537    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-z2tqt" podUID="5ef7993f-e88f-4430-86be-7a8367cdc27c"
Jul 22 18:53:29.460583 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:29.460554    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:53:29.977856 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:53:29.977827    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:53:29.978037 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:53:29.977827    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:53:29.978037 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:53:29.977832    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:53:34.241955 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:53:34.241919    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:53:34.241955 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:53:34.241960    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:53:34.242492 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:53:34.241990    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:53:34.242492 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:34.242054    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:53:34.242492 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:34.242065    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 18:53:34.242492 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:34.242085    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 18:53:34.242492 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:34.242107    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8d4d897-nz2g7: secret "image-registry-tls" not found
Jul 22 18:53:34.242492 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:34.242115    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:36.242099737 +0000 UTC m=+286.274117245 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:53:34.242492 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:34.242128    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls podName:5ef7993f-e88f-4430-86be-7a8367cdc27c nodeName:}" failed. No retries permitted until 2026-07-22 18:55:36.242122091 +0000 UTC m=+286.274139599 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls") pod "dns-default-z2tqt" (UID: "5ef7993f-e88f-4430-86be-7a8367cdc27c") : secret "dns-default-metrics-tls" not found
Jul 22 18:53:34.242492 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:53:34.242167    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls podName:2c835e5f-9588-49c1-a80b-f47b83adb40f nodeName:}" failed. No retries permitted until 2026-07-22 18:55:36.24214962 +0000 UTC m=+286.274167128 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls") pod "image-registry-b8d4d897-nz2g7" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f") : secret "image-registry-tls" not found
Jul 22 18:53:44.446330 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:53:44.446248    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:55:02.211811 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:02.211779    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:55:02.212328 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:02.211924    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 18:55:02.212328 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:02.211995    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs podName:d21e4142-fe9f-48f6-951e-d4520913b26d nodeName:}" failed. No retries permitted until 2026-07-22 18:57:04.211978547 +0000 UTC m=+374.243996060 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs") pod "network-metrics-daemon-fg676" (UID: "d21e4142-fe9f-48f6-951e-d4520913b26d") : secret "metrics-daemon-secret" not found
Jul 22 18:55:21.084998 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.084921    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-6kg4x"]
Jul 22 18:55:21.086524 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.086508    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.089395 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.089376    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-m2w7j\""
Jul 22 18:55:21.089508 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.089434    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 22 18:55:21.090505 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.090488    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 18:55:21.090599 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.090517    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 22 18:55:21.090599 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.090495    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 18:55:21.095330 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.095312    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 22 18:55:21.104108 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.104089    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-6kg4x"]
Jul 22 18:55:21.229258 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.229233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4133-70f4-4a6f-b987-e375778ee50e-tmp\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.229345 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.229266    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bdf4133-70f4-4a6f-b987-e375778ee50e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.229345 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.229286    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bdf4133-70f4-4a6f-b987-e375778ee50e-serving-cert\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.229345 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.229311    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tvzh\" (UniqueName: \"kubernetes.io/projected/2bdf4133-70f4-4a6f-b987-e375778ee50e-kube-api-access-9tvzh\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.229445 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.229418    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bdf4133-70f4-4a6f-b987-e375778ee50e-service-ca-bundle\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.229482 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.229460    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4133-70f4-4a6f-b987-e375778ee50e-snapshots\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.330154 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.330117    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4133-70f4-4a6f-b987-e375778ee50e-tmp\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.330154 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.330154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bdf4133-70f4-4a6f-b987-e375778ee50e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.330291 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.330175    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bdf4133-70f4-4a6f-b987-e375778ee50e-serving-cert\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.330325 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.330292    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tvzh\" (UniqueName: \"kubernetes.io/projected/2bdf4133-70f4-4a6f-b987-e375778ee50e-kube-api-access-9tvzh\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.330369 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.330356    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bdf4133-70f4-4a6f-b987-e375778ee50e-service-ca-bundle\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.330408 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.330393    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4133-70f4-4a6f-b987-e375778ee50e-snapshots\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.330556 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.330533    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4133-70f4-4a6f-b987-e375778ee50e-tmp\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.330940 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.330921    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4133-70f4-4a6f-b987-e375778ee50e-snapshots\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.331024 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.330944    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bdf4133-70f4-4a6f-b987-e375778ee50e-service-ca-bundle\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.331139 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.331122    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bdf4133-70f4-4a6f-b987-e375778ee50e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.332537 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.332512    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bdf4133-70f4-4a6f-b987-e375778ee50e-serving-cert\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.338404 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.338356    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tvzh\" (UniqueName: \"kubernetes.io/projected/2bdf4133-70f4-4a6f-b987-e375778ee50e-kube-api-access-9tvzh\") pod \"insights-operator-555d9d7885-6kg4x\" (UID: \"2bdf4133-70f4-4a6f-b987-e375778ee50e\") " pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.395968 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.395947    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-6kg4x"
Jul 22 18:55:21.512698 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:21.512670    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-6kg4x"]
Jul 22 18:55:21.515558 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:55:21.515529    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bdf4133_70f4_4a6f_b987_e375778ee50e.slice/crio-83e4f0800fce148466e8f1567dc64e1e05195b9bfdc4927ef904b245e04bc2f3 WatchSource:0}: Error finding container 83e4f0800fce148466e8f1567dc64e1e05195b9bfdc4927ef904b245e04bc2f3: Status 404 returned error can't find the container with id 83e4f0800fce148466e8f1567dc64e1e05195b9bfdc4927ef904b245e04bc2f3
Jul 22 18:55:22.234625 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:22.234595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-6kg4x" event={"ID":"2bdf4133-70f4-4a6f-b987-e375778ee50e","Type":"ContainerStarted","Data":"83e4f0800fce148466e8f1567dc64e1e05195b9bfdc4927ef904b245e04bc2f3"}
Jul 22 18:55:24.030333 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.030292    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jzqv4"]
Jul 22 18:55:24.032579 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.032555    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.036318 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.036298    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 18:55:24.036588 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.036570    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rcw4g\""
Jul 22 18:55:24.037309 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.037293    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 18:55:24.051799 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.051776    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jzqv4"]
Jul 22 18:55:24.154025 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.153999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-crio-socket\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.154166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.154048    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.154166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.154085    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.154166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.154105    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8wl2\" (UniqueName: \"kubernetes.io/projected/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-api-access-d8wl2\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.154166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.154129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-data-volume\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.240802 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.240771    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-6kg4x" event={"ID":"2bdf4133-70f4-4a6f-b987-e375778ee50e","Type":"ContainerStarted","Data":"7e9e0484f6e0c8bb2a87f7a6b44a7f089d841ed2877dd6d1f1c0de9547a25b72"}
Jul 22 18:55:24.255232 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.255211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-crio-socket\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.255339 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.255249    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.255339 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.255270    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.255339 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.255295    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wl2\" (UniqueName: \"kubernetes.io/projected/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-api-access-d8wl2\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.255339 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.255331    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-data-volume\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.255483 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.255338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-crio-socket\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.255483 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:24.255366    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:55:24.255483 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:24.255389    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:24.255483 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:24.255432    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:24.755415055 +0000 UTC m=+274.787432584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : configmap "kube-rbac-proxy" not found
Jul 22 18:55:24.255483 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:24.255448    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:24.755440231 +0000 UTC m=+274.787457738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:24.255732 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.255618    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-data-volume\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.267929 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.267905    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8wl2\" (UniqueName: \"kubernetes.io/projected/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-api-access-d8wl2\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.758954 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.758921    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.758954 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:24.758958    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:24.759160 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:24.759040    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:55:24.759160 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:24.759056    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:24.759160 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:24.759106    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:25.759091348 +0000 UTC m=+275.791108856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:24.759160 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:24.759138    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:25.759131237 +0000 UTC m=+275.791148745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : configmap "kube-rbac-proxy" not found
Jul 22 18:55:25.344204 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.344153    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-6kg4x" podStartSLOduration=2.5728003 podStartE2EDuration="4.344136473s" totalImagesPullingTime="1.771336173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:24.256185655 +0000 UTC m=+274.288203186" watchObservedRunningTime="2026-07-22 18:55:25.344136473 +0000 UTC m=+275.376154002"
Jul 22 18:55:25.344940 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.344923    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6"]
Jul 22 18:55:25.346878 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.346864    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6"
Jul 22 18:55:25.350000 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.349979    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 22 18:55:25.350122 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.349979    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lgv4d\""
Jul 22 18:55:25.350796 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.350778    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 22 18:55:25.359172 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.359149    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6"]
Jul 22 18:55:25.464507 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.464486    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf997\" (UniqueName: \"kubernetes.io/projected/9d9c768d-8962-4886-89f1-0411e082417a-kube-api-access-kf997\") pod \"migrator-7cc9c7ffd7-sttj6\" (UID: \"9d9c768d-8962-4886-89f1-0411e082417a\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6"
Jul 22 18:55:25.565427 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.565395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf997\" (UniqueName: \"kubernetes.io/projected/9d9c768d-8962-4886-89f1-0411e082417a-kube-api-access-kf997\") pod \"migrator-7cc9c7ffd7-sttj6\" (UID: \"9d9c768d-8962-4886-89f1-0411e082417a\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6"
Jul 22 18:55:25.573405 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.573384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf997\" (UniqueName: \"kubernetes.io/projected/9d9c768d-8962-4886-89f1-0411e082417a-kube-api-access-kf997\") pod \"migrator-7cc9c7ffd7-sttj6\" (UID: \"9d9c768d-8962-4886-89f1-0411e082417a\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6"
Jul 22 18:55:25.655400 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.655377    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6"
Jul 22 18:55:25.767390 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.767364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:25.767521 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.767459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:25.767563 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:25.767520    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:25.767597 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:25.767582    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:27.76756733 +0000 UTC m=+277.799584841 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:25.767637 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:25.767523    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:55:25.767673 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:25.767649    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:27.767636767 +0000 UTC m=+277.799654276 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : configmap "kube-rbac-proxy" not found
Jul 22 18:55:25.775616 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:25.775507    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6"]
Jul 22 18:55:25.777992 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:55:25.777969    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9c768d_8962_4886_89f1_0411e082417a.slice/crio-fc95212c13fef89dc868439bdb75ca4f1d09b6d9f253d3d8ebb1a5f5222be1aa WatchSource:0}: Error finding container fc95212c13fef89dc868439bdb75ca4f1d09b6d9f253d3d8ebb1a5f5222be1aa: Status 404 returned error can't find the container with id fc95212c13fef89dc868439bdb75ca4f1d09b6d9f253d3d8ebb1a5f5222be1aa
Jul 22 18:55:26.021112 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:26.021035    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l6nb4_a6bbcded-2352-4a2f-ab3a-a6715fabcc1e/dns-node-resolver/0.log"
Jul 22 18:55:26.245809 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:26.245783    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6" event={"ID":"9d9c768d-8962-4886-89f1-0411e082417a","Type":"ContainerStarted","Data":"fc95212c13fef89dc868439bdb75ca4f1d09b6d9f253d3d8ebb1a5f5222be1aa"}
Jul 22 18:55:26.820598 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:26.820572    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-98jjr_2106254d-8297-442a-b586-3b8813e4ab8b/node-ca/0.log"
Jul 22 18:55:27.253054 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:27.253024    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6" event={"ID":"9d9c768d-8962-4886-89f1-0411e082417a","Type":"ContainerStarted","Data":"8e6de3aa3ffd6337b605aa5e4ca9c66c2cc3decc8da12663410fba3ae3ecd4e5"}
Jul 22 18:55:27.253166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:27.253093    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6" event={"ID":"9d9c768d-8962-4886-89f1-0411e082417a","Type":"ContainerStarted","Data":"9954252b5016aa839636f24bfc3698396837fbb52d5474dbbf17e63a3ee5db72"}
Jul 22 18:55:27.268626 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:27.268584    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-sttj6" podStartSLOduration=1.008174267 podStartE2EDuration="2.268571476s" totalImagesPullingTime="1.260397209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:27.267949376 +0000 UTC m=+277.299966946" watchObservedRunningTime="2026-07-22 18:55:27.268571476 +0000 UTC m=+277.300589005"
Jul 22 18:55:27.621810 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:27.621785    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6bpcc_956d7809-f242-41c4-a268-f692575d0b17/serve-healthcheck-canary/0.log"
Jul 22 18:55:27.782981 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:27.782946    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:27.783135 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:27.782988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:27.783135 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:27.783091    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:55:27.783254 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:27.783169    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:31.783148388 +0000 UTC m=+281.815165930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : configmap "kube-rbac-proxy" not found
Jul 22 18:55:27.783254 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:27.783096    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:27.783254 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:27.783232    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:31.783220046 +0000 UTC m=+281.815237555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:31.811254 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:31.811210    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:31.811254 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:31.811261    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:31.811707 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:31.811340    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 18:55:31.811707 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:31.811381    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:31.811707 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:31.811415    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:39.811397335 +0000 UTC m=+289.843414844 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : configmap "kube-rbac-proxy" not found
Jul 22 18:55:31.811707 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:31.811431    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:39.811424585 +0000 UTC m=+289.843442093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:32.978341 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:32.978285    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh" podUID="dfc22088-efc9-4e37-926b-a00989038020"
Jul 22 18:55:32.978341 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:32.978297    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-z2tqt" podUID="5ef7993f-e88f-4430-86be-7a8367cdc27c"
Jul 22 18:55:32.978341 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:32.978305    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7" podUID="2c835e5f-9588-49c1-a80b-f47b83adb40f"
Jul 22 18:55:33.270451 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:33.270375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:55:33.270584 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:33.270375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:55:33.270584 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:33.270375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:55:36.244584 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.244551    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:55:36.244584 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.244590    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:55:36.245013 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.244613    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:55:36.245013 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:36.244767    2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 22 18:55:36.245013 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:36.244839    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert podName:dfc22088-efc9-4e37-926b-a00989038020 nodeName:}" failed. No retries permitted until 2026-07-22 18:57:38.244819811 +0000 UTC m=+408.276837329 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4qrhh" (UID: "dfc22088-efc9-4e37-926b-a00989038020") : secret "networking-console-plugin-cert" not found
Jul 22 18:55:36.246762 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.246734    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ef7993f-e88f-4430-86be-7a8367cdc27c-metrics-tls\") pod \"dns-default-z2tqt\" (UID: \"5ef7993f-e88f-4430-86be-7a8367cdc27c\") " pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:55:36.246939 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.246919    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"image-registry-b8d4d897-nz2g7\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") " pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:55:36.273903 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.273880    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d66st\""
Jul 22 18:55:36.274710 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.274696    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-phz5q\""
Jul 22 18:55:36.281518 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.281498    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:55:36.281622 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.281525    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:55:36.409357 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.409330    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b8d4d897-nz2g7"]
Jul 22 18:55:36.412314 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:55:36.412277    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c835e5f_9588_49c1_a80b_f47b83adb40f.slice/crio-518ebf65239e14e78404f2fa1171755d66029b8816b755146c37dd345a165acd WatchSource:0}: Error finding container 518ebf65239e14e78404f2fa1171755d66029b8816b755146c37dd345a165acd: Status 404 returned error can't find the container with id 518ebf65239e14e78404f2fa1171755d66029b8816b755146c37dd345a165acd
Jul 22 18:55:36.422811 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:36.422788    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z2tqt"]
Jul 22 18:55:36.425136 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:55:36.425116    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef7993f_e88f_4430_86be_7a8367cdc27c.slice/crio-38783cb7e9a6c336a0db33cf5b36ff2522c3299bac3c607f39a1dd6848dc7734 WatchSource:0}: Error finding container 38783cb7e9a6c336a0db33cf5b36ff2522c3299bac3c607f39a1dd6848dc7734: Status 404 returned error can't find the container with id 38783cb7e9a6c336a0db33cf5b36ff2522c3299bac3c607f39a1dd6848dc7734
Jul 22 18:55:37.286998 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:37.286963    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7" event={"ID":"2c835e5f-9588-49c1-a80b-f47b83adb40f","Type":"ContainerStarted","Data":"7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f"}
Jul 22 18:55:37.287426 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:37.287051    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7" event={"ID":"2c835e5f-9588-49c1-a80b-f47b83adb40f","Type":"ContainerStarted","Data":"518ebf65239e14e78404f2fa1171755d66029b8816b755146c37dd345a165acd"}
Jul 22 18:55:37.287426 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:37.287071    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:55:37.288107 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:37.288081    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z2tqt" event={"ID":"5ef7993f-e88f-4430-86be-7a8367cdc27c","Type":"ContainerStarted","Data":"38783cb7e9a6c336a0db33cf5b36ff2522c3299bac3c607f39a1dd6848dc7734"}
Jul 22 18:55:37.308026 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:37.307970    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7" podStartSLOduration=286.307952909 podStartE2EDuration="4m46.307952909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:37.306091973 +0000 UTC m=+287.338109505" watchObservedRunningTime="2026-07-22 18:55:37.307952909 +0000 UTC m=+287.339970417"
Jul 22 18:55:38.292447 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:38.292418    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z2tqt" event={"ID":"5ef7993f-e88f-4430-86be-7a8367cdc27c","Type":"ContainerStarted","Data":"3a7fc5595e0c7c781fc20051f4e99856f282fd35c696a23c98d9648896b3f43c"}
Jul 22 18:55:38.292821 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:38.292473    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z2tqt" event={"ID":"5ef7993f-e88f-4430-86be-7a8367cdc27c","Type":"ContainerStarted","Data":"309bd5001ee09a4f48450959d6546e3e1d740c7af8af8f4d3537c5b45be8571b"}
Jul 22 18:55:38.309584 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:38.309539    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z2tqt" podStartSLOduration=250.651433146 podStartE2EDuration="4m12.309524248s" totalImagesPullingTime="1.658091102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:38.308961966 +0000 UTC m=+288.340979507" watchObservedRunningTime="2026-07-22 18:55:38.309524248 +0000 UTC m=+288.341541778"
Jul 22 18:55:39.295506 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:39.295473    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:55:39.872221 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:39.872178    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:39.872400 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:39.872225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:39.872400 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:39.872355    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:39.872516 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:39.872419    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls podName:6683dc1e-7f60-4af9-aae1-dd0b2848b9ac nodeName:}" failed. No retries permitted until 2026-07-22 18:55:55.872402028 +0000 UTC m=+305.904419538 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzqv4" (UID: "6683dc1e-7f60-4af9-aae1-dd0b2848b9ac") : secret "insights-runtime-extractor-tls" not found
Jul 22 18:55:39.872792 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:39.872769    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:47.447519 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:47.447460    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fg676" podUID="d21e4142-fe9f-48f6-951e-d4520913b26d"
Jul 22 18:55:49.300151 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:49.300122    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z2tqt"
Jul 22 18:55:50.282060 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:50.282032    2558 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 18:55:55.669674 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:55.669640    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"]
Jul 22 18:55:55.674359 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:55.674344    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"
Jul 22 18:55:55.676846 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:55.676824    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 22 18:55:55.676957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:55.676824    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-5mv5q\""
Jul 22 18:55:55.680634 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:55.680547    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"]
Jul 22 18:55:55.778424 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:55.778399    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/421cdc54-b59d-47d0-a379-0eb8d0325610-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wwl77\" (UID: \"421cdc54-b59d-47d0-a379-0eb8d0325610\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"
Jul 22 18:55:55.878827 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:55.878788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/421cdc54-b59d-47d0-a379-0eb8d0325610-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wwl77\" (UID: \"421cdc54-b59d-47d0-a379-0eb8d0325610\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"
Jul 22 18:55:55.878950 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:55.878884    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:55.878950 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:55.878921    2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 22 18:55:55.879059 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:55:55.878988    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/421cdc54-b59d-47d0-a379-0eb8d0325610-tls-certificates podName:421cdc54-b59d-47d0-a379-0eb8d0325610 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:56.378972259 +0000 UTC m=+306.410989767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/421cdc54-b59d-47d0-a379-0eb8d0325610-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-wwl77" (UID: "421cdc54-b59d-47d0-a379-0eb8d0325610") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 22 18:55:55.881209 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:55.881177    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6683dc1e-7f60-4af9-aae1-dd0b2848b9ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzqv4\" (UID: \"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac\") " pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:56.143704 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.143683    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rcw4g\""
Jul 22 18:55:56.152153 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.152136    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jzqv4"
Jul 22 18:55:56.272142 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.272120    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jzqv4"]
Jul 22 18:55:56.274361 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:55:56.274335    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6683dc1e_7f60_4af9_aae1_dd0b2848b9ac.slice/crio-f63b86d5a903b88cf2de8e8c6ad5ca0b4ec50f8adfd0fcadb6d6bf2aa0291ede WatchSource:0}: Error finding container f63b86d5a903b88cf2de8e8c6ad5ca0b4ec50f8adfd0fcadb6d6bf2aa0291ede: Status 404 returned error can't find the container with id f63b86d5a903b88cf2de8e8c6ad5ca0b4ec50f8adfd0fcadb6d6bf2aa0291ede
Jul 22 18:55:56.275693 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.275670    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 18:55:56.285524 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.285500    2558 patch_prober.go:28] interesting pod/image-registry-b8d4d897-nz2g7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 22 18:55:56.285617 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.285540    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7" podUID="2c835e5f-9588-49c1-a80b-f47b83adb40f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 18:55:56.349643 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.349621    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzqv4" event={"ID":"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac","Type":"ContainerStarted","Data":"f63b86d5a903b88cf2de8e8c6ad5ca0b4ec50f8adfd0fcadb6d6bf2aa0291ede"}
Jul 22 18:55:56.382125 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.382101    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/421cdc54-b59d-47d0-a379-0eb8d0325610-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wwl77\" (UID: \"421cdc54-b59d-47d0-a379-0eb8d0325610\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"
Jul 22 18:55:56.384377 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.384354    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/421cdc54-b59d-47d0-a379-0eb8d0325610-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wwl77\" (UID: \"421cdc54-b59d-47d0-a379-0eb8d0325610\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"
Jul 22 18:55:56.583784 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.583677    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"
Jul 22 18:55:56.712998 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:56.712965    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"]
Jul 22 18:55:56.715708 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:55:56.715681    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421cdc54_b59d_47d0_a379_0eb8d0325610.slice/crio-5d273236090f53c7861ec0674b5ac404f560a6949c1056e1c000ab42c037c2ac WatchSource:0}: Error finding container 5d273236090f53c7861ec0674b5ac404f560a6949c1056e1c000ab42c037c2ac: Status 404 returned error can't find the container with id 5d273236090f53c7861ec0674b5ac404f560a6949c1056e1c000ab42c037c2ac
Jul 22 18:55:57.355235 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:57.355197    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77" event={"ID":"421cdc54-b59d-47d0-a379-0eb8d0325610","Type":"ContainerStarted","Data":"5d273236090f53c7861ec0674b5ac404f560a6949c1056e1c000ab42c037c2ac"}
Jul 22 18:55:58.297450 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.297373    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:55:58.360995 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.360961    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77" event={"ID":"421cdc54-b59d-47d0-a379-0eb8d0325610","Type":"ContainerStarted","Data":"f8b2a77233a790bfac856f4edc25e53160a745b434ceb9ce0f01845cf4f7eed1"}
Jul 22 18:55:58.361166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.361129    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"
Jul 22 18:55:58.366713 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.366688    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77"
Jul 22 18:55:58.376408 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.376358    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wwl77" podStartSLOduration=2.158331639 podStartE2EDuration="3.376339431s" totalImagesPullingTime="1.218007792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:58.375327137 +0000 UTC m=+308.407344668" watchObservedRunningTime="2026-07-22 18:55:58.376339431 +0000 UTC m=+308.408356962"
Jul 22 18:55:58.445378 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.445348    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:55:58.729141 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.729113    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"]
Jul 22 18:55:58.735602 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.735579    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:58.738305 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.738269    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 22 18:55:58.738437 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.738276    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 22 18:55:58.738437 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.738371    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 18:55:58.738437 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.738383    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 18:55:58.738616 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.738375    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bcrgb\""
Jul 22 18:55:58.739250 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.739223    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 18:55:58.742822 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.742800    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"]
Jul 22 18:55:58.904507 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.904470    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d9512c06-e290-4738-91cc-93fe55610fd6-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:58.904683 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.904512    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9512c06-e290-4738-91cc-93fe55610fd6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:58.904683 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.904555    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9512c06-e290-4738-91cc-93fe55610fd6-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:58.904683 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:58.904611    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ppgk\" (UniqueName: \"kubernetes.io/projected/d9512c06-e290-4738-91cc-93fe55610fd6-kube-api-access-5ppgk\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:59.005579 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:59.005497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9512c06-e290-4738-91cc-93fe55610fd6-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:59.005579 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:59.005535    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ppgk\" (UniqueName: \"kubernetes.io/projected/d9512c06-e290-4738-91cc-93fe55610fd6-kube-api-access-5ppgk\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:59.005822 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:59.005631    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d9512c06-e290-4738-91cc-93fe55610fd6-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:59.005822 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:59.005664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9512c06-e290-4738-91cc-93fe55610fd6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:59.006391 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:59.006333    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9512c06-e290-4738-91cc-93fe55610fd6-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:59.008695 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:59.008644    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9512c06-e290-4738-91cc-93fe55610fd6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:59.008820 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:59.008672    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d9512c06-e290-4738-91cc-93fe55610fd6-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:59.013876 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:59.013833    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ppgk\" (UniqueName: \"kubernetes.io/projected/d9512c06-e290-4738-91cc-93fe55610fd6-kube-api-access-5ppgk\") pod \"prometheus-operator-5945b5cfd-lx2gf\" (UID: \"d9512c06-e290-4738-91cc-93fe55610fd6\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:55:59.046209 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:55:59.046186    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"
Jul 22 18:56:01.226758 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:01.226717    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf"]
Jul 22 18:56:01.229602 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:56:01.229575    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9512c06_e290_4738_91cc_93fe55610fd6.slice/crio-d16092bb33a9710fbd720f8e24f862e6653103741cea66ed93a249edaeee123a WatchSource:0}: Error finding container d16092bb33a9710fbd720f8e24f862e6653103741cea66ed93a249edaeee123a: Status 404 returned error can't find the container with id d16092bb33a9710fbd720f8e24f862e6653103741cea66ed93a249edaeee123a
Jul 22 18:56:01.372228 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:01.372136    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzqv4" event={"ID":"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac","Type":"ContainerStarted","Data":"0ad13e1258e35126b179097f2e21721ad1c27266181cae0781c2c8cb50e3b575"}
Jul 22 18:56:01.373262 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:01.373239    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf" event={"ID":"d9512c06-e290-4738-91cc-93fe55610fd6","Type":"ContainerStarted","Data":"d16092bb33a9710fbd720f8e24f862e6653103741cea66ed93a249edaeee123a"}
Jul 22 18:56:03.383152 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:03.383111    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf" event={"ID":"d9512c06-e290-4738-91cc-93fe55610fd6","Type":"ContainerStarted","Data":"7f1137adb0755275f94313ce02043c4f93b118e5175748f7e2c409a627cb6502"}
Jul 22 18:56:03.383615 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:03.383176    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf" event={"ID":"d9512c06-e290-4738-91cc-93fe55610fd6","Type":"ContainerStarted","Data":"169748a8b646dd490453d22eed047404039bb8d2576bb57bdd2b7c95c0271051"}
Jul 22 18:56:03.402715 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:03.402672    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lx2gf" podStartSLOduration=4.017269423 podStartE2EDuration="5.402656621s" totalImagesPullingTime="1.385387198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:03.400976688 +0000 UTC m=+313.432994240" watchObservedRunningTime="2026-07-22 18:56:03.402656621 +0000 UTC m=+313.434674150"
Jul 22 18:56:05.154692 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.154623    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z5hdr"]
Jul 22 18:56:05.157832 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.157815    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.160892 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.160862    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 18:56:05.161022 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.160929    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 18:56:05.161079 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.161026    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 18:56:05.161079 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.161038    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gzn74\""
Jul 22 18:56:05.254563 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.254540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-tls\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.254673 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.254567    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-root\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.254673 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.254591    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-sys\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.254673 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.254608    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-accelerators-collector-config\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.254673 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.254630    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-textfile\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.254673 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.254664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz695\" (UniqueName: \"kubernetes.io/projected/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-kube-api-access-tz695\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.254889 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.254711    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.254889 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.254744    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-metrics-client-ca\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.254889 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.254789    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-wtmp\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.355555 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355529    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-sys\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.355662 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355568    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-accelerators-collector-config\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.355662 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-textfile\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.355662 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355624    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz695\" (UniqueName: \"kubernetes.io/projected/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-kube-api-access-tz695\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.355662 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355641    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-sys\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.355917 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355663    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.355917 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-metrics-client-ca\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.355917 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355912    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-wtmp\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.356065 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355961    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-tls\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.356065 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.355985    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-root\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.356065 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.356017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-textfile\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.356218 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.356066    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-root\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.356218 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.356098    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-wtmp\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.356218 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:56:05.356122    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 22 18:56:05.356218 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:56:05.356183    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-tls podName:6dd1d8be-7df1-42a6-bd54-328cc22ea5b5 nodeName:}" failed. No retries permitted until 2026-07-22 18:56:05.856163882 +0000 UTC m=+315.888181394 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-tls") pod "node-exporter-z5hdr" (UID: "6dd1d8be-7df1-42a6-bd54-328cc22ea5b5") : secret "node-exporter-tls" not found
Jul 22 18:56:05.356369 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.356239    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-accelerators-collector-config\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.356456 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.356424    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-metrics-client-ca\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.358395 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.358377    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.366459 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.366440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz695\" (UniqueName: \"kubernetes.io/projected/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-kube-api-access-tz695\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.392088 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.392060    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzqv4" event={"ID":"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac","Type":"ContainerStarted","Data":"d520c6d4f72e9cba399771ce1e0345f27869d54d86e565db8e2c9c1b35a43938"}
Jul 22 18:56:05.859480 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.859448    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-tls\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:05.862334 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:05.862304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6dd1d8be-7df1-42a6-bd54-328cc22ea5b5-node-exporter-tls\") pod \"node-exporter-z5hdr\" (UID: \"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5\") " pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:06.067518 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.066645    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z5hdr"
Jul 22 18:56:06.084504 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:56:06.083279    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd1d8be_7df1_42a6_bd54_328cc22ea5b5.slice/crio-b41b6c4193e3123c270d798caede4cfcc913b09722a6487cb0b98b1484c7e35d WatchSource:0}: Error finding container b41b6c4193e3123c270d798caede4cfcc913b09722a6487cb0b98b1484c7e35d: Status 404 returned error can't find the container with id b41b6c4193e3123c270d798caede4cfcc913b09722a6487cb0b98b1484c7e35d
Jul 22 18:56:06.209709 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.209639    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 18:56:06.214816 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.214791    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.217540 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.217514    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6hpb2\""
Jul 22 18:56:06.217540 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.217530    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 22 18:56:06.217721 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.217674    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 22 18:56:06.217905 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.217809    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 22 18:56:06.218287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.218125    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 22 18:56:06.218287 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.218175    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 22 18:56:06.218444 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.218373    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 22 18:56:06.218554 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.218530    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 22 18:56:06.218772 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.218727    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 22 18:56:06.218977 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.218961    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 22 18:56:06.229996 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.229974    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 18:56:06.264649 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264620    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-web-config\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.264797 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264677    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.264797 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264708    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-volume\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.264797 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.264797 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264787    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.265003 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264816    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-out\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.265003 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264858    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.265003 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264878    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.265003 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264895    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sg9r\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-kube-api-access-2sg9r\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.265003 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264942    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.265003 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.264974    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.265209 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.265004    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.265209 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.265030    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366006 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.365980    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366021    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-web-config\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366085    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-volume\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366163    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366375 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366191    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366375 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366218    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-out\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366375 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366375 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366271    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366375 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366295    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sg9r\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-kube-api-access-2sg9r\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366375 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366681 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366681 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.366681 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:56:06.366493    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-trusted-ca-bundle podName:86feaef0-8bc9-4342-abc2-77fcee65ffdd nodeName:}" failed. No retries permitted until 2026-07-22 18:56:06.866475518 +0000 UTC m=+316.898493037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd") : configmap references non-existent config key: ca-bundle.crt
Jul 22 18:56:06.366681 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:56:06.366571    2558 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 22 18:56:06.366681 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:56:06.366619    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-main-tls podName:86feaef0-8bc9-4342-abc2-77fcee65ffdd nodeName:}" failed. No retries permitted until 2026-07-22 18:56:06.866605102 +0000 UTC m=+316.898622623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd") : secret "alertmanager-main-tls" not found
Jul 22 18:56:06.367193 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.366944    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.367925 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.367901    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.369418 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.369393    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.370029 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.370005    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-out\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.370121 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.370087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-volume\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.370503 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.370480    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-web-config\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.371032 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.370994    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.371338 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.371320    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.371525 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.371502    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.371591 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.371536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.375649 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.375627    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sg9r\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-kube-api-access-2sg9r\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.397422 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.397394    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z5hdr" event={"ID":"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5","Type":"ContainerStarted","Data":"b41b6c4193e3123c270d798caede4cfcc913b09722a6487cb0b98b1484c7e35d"}
Jul 22 18:56:06.870839 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.870798    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.871013 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.870845    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.871629 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.871595    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:06.873447 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:06.873426    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:07.128150 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:07.128071    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:56:08.797818 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:08.797795    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 18:56:08.799585 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:56:08.799559    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86feaef0_8bc9_4342_abc2_77fcee65ffdd.slice/crio-50a00aa6bc583b3ecb10b50714896aaf6fa96efe5b2dd525bdf2ce015b59e532 WatchSource:0}: Error finding container 50a00aa6bc583b3ecb10b50714896aaf6fa96efe5b2dd525bdf2ce015b59e532: Status 404 returned error can't find the container with id 50a00aa6bc583b3ecb10b50714896aaf6fa96efe5b2dd525bdf2ce015b59e532
Jul 22 18:56:09.408486 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.408452    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerStarted","Data":"50a00aa6bc583b3ecb10b50714896aaf6fa96efe5b2dd525bdf2ce015b59e532"}
Jul 22 18:56:09.410896 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.410865    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzqv4" event={"ID":"6683dc1e-7f60-4af9-aae1-dd0b2848b9ac","Type":"ContainerStarted","Data":"65dab56ad9d022c694f53ce67be4a4c1da064339f2c75b2ab63d8ea6aaee7c2c"}
Jul 22 18:56:09.412576 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.412548    2558 generic.go:356] "Generic (PLEG): container finished" podID="6dd1d8be-7df1-42a6-bd54-328cc22ea5b5" containerID="aac0ad83aecbefedfcb07dd55e81652179ee3d8862abd45499348e1eb32d3df1" exitCode=0
Jul 22 18:56:09.412699 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.412585    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z5hdr" event={"ID":"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5","Type":"ContainerDied","Data":"aac0ad83aecbefedfcb07dd55e81652179ee3d8862abd45499348e1eb32d3df1"}
Jul 22 18:56:09.429826 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.429779    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jzqv4" podStartSLOduration=33.159707467 podStartE2EDuration="45.429758544s" totalImagesPullingTime="12.270051077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:55:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:09.428603722 +0000 UTC m=+319.460621253" watchObservedRunningTime="2026-07-22 18:56:09.429758544 +0000 UTC m=+319.461776077"
Jul 22 18:56:09.882477 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.882448    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"]
Jul 22 18:56:09.885738 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.885716    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:09.888423 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.888403    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cm24v\""
Jul 22 18:56:09.888547 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.888401    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 22 18:56:09.894080 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.894055    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ccv2\" (UniqueName: \"kubernetes.io/projected/8e27f963-59ee-49df-ad95-64ae9ff428bf-kube-api-access-9ccv2\") pod \"monitoring-plugin-7b7b7bc8bf-rchnt\" (UID: \"8e27f963-59ee-49df-ad95-64ae9ff428bf\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:09.894213 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.894171    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8e27f963-59ee-49df-ad95-64ae9ff428bf-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-rchnt\" (UID: \"8e27f963-59ee-49df-ad95-64ae9ff428bf\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:09.894945 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.894922    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"]
Jul 22 18:56:09.994996 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.994969    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8e27f963-59ee-49df-ad95-64ae9ff428bf-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-rchnt\" (UID: \"8e27f963-59ee-49df-ad95-64ae9ff428bf\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:09.995106 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.995088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ccv2\" (UniqueName: \"kubernetes.io/projected/8e27f963-59ee-49df-ad95-64ae9ff428bf-kube-api-access-9ccv2\") pod \"monitoring-plugin-7b7b7bc8bf-rchnt\" (UID: \"8e27f963-59ee-49df-ad95-64ae9ff428bf\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:09.997132 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:09.997114    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8e27f963-59ee-49df-ad95-64ae9ff428bf-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-rchnt\" (UID: \"8e27f963-59ee-49df-ad95-64ae9ff428bf\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:10.004305 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.004283    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ccv2\" (UniqueName: \"kubernetes.io/projected/8e27f963-59ee-49df-ad95-64ae9ff428bf-kube-api-access-9ccv2\") pod \"monitoring-plugin-7b7b7bc8bf-rchnt\" (UID: \"8e27f963-59ee-49df-ad95-64ae9ff428bf\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:10.195025 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.194966    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:10.316293 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.316267    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"]
Jul 22 18:56:10.319365 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:56:10.319341    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e27f963_59ee_49df_ad95_64ae9ff428bf.slice/crio-07311e449a6a4174bc08b83b249f016a87883b55cf93dbbb11f6274a7b9918d2 WatchSource:0}: Error finding container 07311e449a6a4174bc08b83b249f016a87883b55cf93dbbb11f6274a7b9918d2: Status 404 returned error can't find the container with id 07311e449a6a4174bc08b83b249f016a87883b55cf93dbbb11f6274a7b9918d2
Jul 22 18:56:10.418222 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.418196    2558 generic.go:356] "Generic (PLEG): container finished" podID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerID="d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace" exitCode=0
Jul 22 18:56:10.418316 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.418236    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerDied","Data":"d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace"}
Jul 22 18:56:10.420589 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.420566    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z5hdr" event={"ID":"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5","Type":"ContainerStarted","Data":"ed07145c35d65d1fb37f72eb1c3f2dd67ad021b3578145384af1d9e82adb7b8a"}
Jul 22 18:56:10.420695 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.420621    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z5hdr" event={"ID":"6dd1d8be-7df1-42a6-bd54-328cc22ea5b5","Type":"ContainerStarted","Data":"f71ed7179ef923f27c34452eeb2d6f1ac16b275e68f0f4400d61abee05ab31a4"}
Jul 22 18:56:10.421734 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.421712    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt" event={"ID":"8e27f963-59ee-49df-ad95-64ae9ff428bf","Type":"ContainerStarted","Data":"07311e449a6a4174bc08b83b249f016a87883b55cf93dbbb11f6274a7b9918d2"}
Jul 22 18:56:10.464655 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.464588    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z5hdr" podStartSLOduration=2.826852762 podStartE2EDuration="5.464575482s" totalImagesPullingTime="2.571870353s" totalInitContainerRuntime="65.852367ms" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:10.46382167 +0000 UTC m=+320.495839201" watchObservedRunningTime="2026-07-22 18:56:10.464575482 +0000 UTC m=+320.496593012"
Jul 22 18:56:10.918825 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:10.918780    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b8d4d897-nz2g7"]
Jul 22 18:56:12.431553 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:12.431518    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt" event={"ID":"8e27f963-59ee-49df-ad95-64ae9ff428bf","Type":"ContainerStarted","Data":"d35359b2b93f3b0e2defacab1e0c5e646c7b2341058842f99af7f40e2002f522"}
Jul 22 18:56:12.431986 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:12.431583    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:12.434868 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:12.434844    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerStarted","Data":"835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51"}
Jul 22 18:56:12.434963 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:12.434906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerStarted","Data":"76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab"}
Jul 22 18:56:12.434963 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:12.434933    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerStarted","Data":"eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a"}
Jul 22 18:56:12.434963 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:12.434958    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerStarted","Data":"481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299"}
Jul 22 18:56:12.435089 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:12.434980    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerStarted","Data":"7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a"}
Jul 22 18:56:12.436809 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:12.436792    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt"
Jul 22 18:56:12.448978 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:12.448927    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-rchnt" podStartSLOduration=1.715069857 podStartE2EDuration="3.44891176s" totalImagesPullingTime="1.733841903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:12.448854895 +0000 UTC m=+322.480872435" watchObservedRunningTime="2026-07-22 18:56:12.44891176 +0000 UTC m=+322.480929290"
Jul 22 18:56:13.445418 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:13.445328    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerStarted","Data":"e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4"}
Jul 22 18:56:13.474026 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:13.473983    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.868797915 podStartE2EDuration="7.473969253s" totalImagesPullingTime="3.588647621s" totalInitContainerRuntime="16.523717ms" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:13.471622291 +0000 UTC m=+323.503639822" watchObservedRunningTime="2026-07-22 18:56:13.473969253 +0000 UTC m=+323.505986783"
Jul 22 18:56:35.947162 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:35.947095    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7" podUID="2c835e5f-9588-49c1-a80b-f47b83adb40f" containerName="registry" containerID="cri-o://7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f" gracePeriod=30
Jul 22 18:56:36.187936 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.187911    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:56:36.277078 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277000    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-image-registry-private-configuration\") pod \"2c835e5f-9588-49c1-a80b-f47b83adb40f\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") "
Jul 22 18:56:36.277078 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277036    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-certificates\") pod \"2c835e5f-9588-49c1-a80b-f47b83adb40f\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") "
Jul 22 18:56:36.277078 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277072    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-855lh\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-kube-api-access-855lh\") pod \"2c835e5f-9588-49c1-a80b-f47b83adb40f\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") "
Jul 22 18:56:36.277330 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277101    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-trusted-ca\") pod \"2c835e5f-9588-49c1-a80b-f47b83adb40f\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") "
Jul 22 18:56:36.277330 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277122    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-bound-sa-token\") pod \"2c835e5f-9588-49c1-a80b-f47b83adb40f\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") "
Jul 22 18:56:36.277330 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277205    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2c835e5f-9588-49c1-a80b-f47b83adb40f-ca-trust-extracted\") pod \"2c835e5f-9588-49c1-a80b-f47b83adb40f\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") "
Jul 22 18:56:36.277330 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277242    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-installation-pull-secrets\") pod \"2c835e5f-9588-49c1-a80b-f47b83adb40f\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") "
Jul 22 18:56:36.277330 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277295    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") pod \"2c835e5f-9588-49c1-a80b-f47b83adb40f\" (UID: \"2c835e5f-9588-49c1-a80b-f47b83adb40f\") "
Jul 22 18:56:36.277605 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277576    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2c835e5f-9588-49c1-a80b-f47b83adb40f" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 18:56:36.278011 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.277978    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2c835e5f-9588-49c1-a80b-f47b83adb40f" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 18:56:36.279588 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.279537    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2c835e5f-9588-49c1-a80b-f47b83adb40f" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:56:36.279683 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.279582    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2c835e5f-9588-49c1-a80b-f47b83adb40f" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:56:36.279775 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.279706    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2c835e5f-9588-49c1-a80b-f47b83adb40f" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:56:36.279775 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.279743    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2c835e5f-9588-49c1-a80b-f47b83adb40f" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:56:36.279889 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.279810    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-kube-api-access-855lh" (OuterVolumeSpecName: "kube-api-access-855lh") pod "2c835e5f-9588-49c1-a80b-f47b83adb40f" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f"). InnerVolumeSpecName "kube-api-access-855lh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:56:36.285847 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.285802    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c835e5f-9588-49c1-a80b-f47b83adb40f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2c835e5f-9588-49c1-a80b-f47b83adb40f" (UID: "2c835e5f-9588-49c1-a80b-f47b83adb40f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 18:56:36.378035 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.378015    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2c835e5f-9588-49c1-a80b-f47b83adb40f-ca-trust-extracted\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:56:36.378035 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.378034    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-installation-pull-secrets\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:56:36.378134 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.378045    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:56:36.378134 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.378055    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2c835e5f-9588-49c1-a80b-f47b83adb40f-image-registry-private-configuration\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:56:36.378134 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.378064    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-registry-certificates\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:56:36.378134 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.378073    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-855lh\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-kube-api-access-855lh\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:56:36.378134 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.378082    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2c835e5f-9588-49c1-a80b-f47b83adb40f-trusted-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:56:36.378134 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.378089    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c835e5f-9588-49c1-a80b-f47b83adb40f-bound-sa-token\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:56:36.526370 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.526339    2558 generic.go:356] "Generic (PLEG): container finished" podID="2c835e5f-9588-49c1-a80b-f47b83adb40f" containerID="7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f" exitCode=0
Jul 22 18:56:36.526458 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.526389    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7"
Jul 22 18:56:36.526458 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.526425    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7" event={"ID":"2c835e5f-9588-49c1-a80b-f47b83adb40f","Type":"ContainerDied","Data":"7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f"}
Jul 22 18:56:36.526526 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.526485    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8d4d897-nz2g7" event={"ID":"2c835e5f-9588-49c1-a80b-f47b83adb40f","Type":"ContainerDied","Data":"518ebf65239e14e78404f2fa1171755d66029b8816b755146c37dd345a165acd"}
Jul 22 18:56:36.526526 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.526504    2558 scope.go:117] "RemoveContainer" containerID="7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f"
Jul 22 18:56:36.535912 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.535891    2558 scope.go:117] "RemoveContainer" containerID="7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f"
Jul 22 18:56:36.536145 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:56:36.536127    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f\": container with ID starting with 7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f not found: ID does not exist" containerID="7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f"
Jul 22 18:56:36.536196 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.536152    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f"} err="failed to get container status \"7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f\": rpc error: code = NotFound desc = could not find container \"7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f\": container with ID starting with 7479f73503f518732aedfaa4838d1499413d0ba71a5076289c6fc9122a36bb3f not found: ID does not exist"
Jul 22 18:56:36.543042 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.543022    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b8d4d897-nz2g7"]
Jul 22 18:56:36.548738 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:36.548714    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b8d4d897-nz2g7"]
Jul 22 18:56:38.449547 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:38.449515    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c835e5f-9588-49c1-a80b-f47b83adb40f" path="/var/lib/kubelet/pods/2c835e5f-9588-49c1-a80b-f47b83adb40f/volumes"
Jul 22 18:56:44.553914 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:44.553880    2558 generic.go:356] "Generic (PLEG): container finished" podID="2bdf4133-70f4-4a6f-b987-e375778ee50e" containerID="7e9e0484f6e0c8bb2a87f7a6b44a7f089d841ed2877dd6d1f1c0de9547a25b72" exitCode=0
Jul 22 18:56:44.554267 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:44.553934    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-6kg4x" event={"ID":"2bdf4133-70f4-4a6f-b987-e375778ee50e","Type":"ContainerDied","Data":"7e9e0484f6e0c8bb2a87f7a6b44a7f089d841ed2877dd6d1f1c0de9547a25b72"}
Jul 22 18:56:44.554267 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:44.554219    2558 scope.go:117] "RemoveContainer" containerID="7e9e0484f6e0c8bb2a87f7a6b44a7f089d841ed2877dd6d1f1c0de9547a25b72"
Jul 22 18:56:45.560968 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:56:45.560931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-6kg4x" event={"ID":"2bdf4133-70f4-4a6f-b987-e375778ee50e","Type":"ContainerStarted","Data":"541e8d0977debc53cded8809ee17c52f86f1daadb48fc6d44c5ea1c1525d93f5"}
Jul 22 18:57:04.282473 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:04.282435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:57:04.287911 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:04.285724    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d21e4142-fe9f-48f6-951e-d4520913b26d-metrics-certs\") pod \"network-metrics-daemon-fg676\" (UID: \"d21e4142-fe9f-48f6-951e-d4520913b26d\") " pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:57:04.448857 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:04.448830    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pz9g5\""
Jul 22 18:57:04.456975 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:04.456952    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fg676"
Jul 22 18:57:04.592540 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:04.592515    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fg676"]
Jul 22 18:57:04.594835 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:57:04.594811    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21e4142_fe9f_48f6_951e_d4520913b26d.slice/crio-a108ee6d14a9cd9da8dd8304ce744d184fb7eb9ce6d1d3cafa250ca60da477c6 WatchSource:0}: Error finding container a108ee6d14a9cd9da8dd8304ce744d184fb7eb9ce6d1d3cafa250ca60da477c6: Status 404 returned error can't find the container with id a108ee6d14a9cd9da8dd8304ce744d184fb7eb9ce6d1d3cafa250ca60da477c6
Jul 22 18:57:04.628250 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:04.628221    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fg676" event={"ID":"d21e4142-fe9f-48f6-951e-d4520913b26d","Type":"ContainerStarted","Data":"a108ee6d14a9cd9da8dd8304ce744d184fb7eb9ce6d1d3cafa250ca60da477c6"}
Jul 22 18:57:05.633829 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:05.633798    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fg676" event={"ID":"d21e4142-fe9f-48f6-951e-d4520913b26d","Type":"ContainerStarted","Data":"331681220f99982027960964761142a21f905cad035b996ab97481a63b153afb"}
Jul 22 18:57:06.638127 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:06.638091    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fg676" event={"ID":"d21e4142-fe9f-48f6-951e-d4520913b26d","Type":"ContainerStarted","Data":"f88b129de8b2f1cecaed5d86ad5258074e87d0a3390810034c01af24777c408c"}
Jul 22 18:57:06.654962 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:06.654911    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fg676" podStartSLOduration=375.711508339 podStartE2EDuration="6m16.654896825s" totalImagesPullingTime="943.388486ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:06.653020392 +0000 UTC m=+376.685037922" watchObservedRunningTime="2026-07-22 18:57:06.654896825 +0000 UTC m=+376.686914377"
Jul 22 18:57:25.431430 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.431398    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 18:57:25.431898 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.431787    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="alertmanager" containerID="cri-o://7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a" gracePeriod=120
Jul 22 18:57:25.431898 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.431820    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy-metric" containerID="cri-o://835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51" gracePeriod=120
Jul 22 18:57:25.431898 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.431845    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy-web" containerID="cri-o://eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a" gracePeriod=120
Jul 22 18:57:25.431898 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.431869    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy" containerID="cri-o://76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab" gracePeriod=120
Jul 22 18:57:25.432100 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.431884    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="prom-label-proxy" containerID="cri-o://e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4" gracePeriod=120
Jul 22 18:57:25.432100 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.431893    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="config-reloader" containerID="cri-o://481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299" gracePeriod=120
Jul 22 18:57:25.710338 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.710255    2558 generic.go:356] "Generic (PLEG): container finished" podID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerID="e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4" exitCode=0
Jul 22 18:57:25.710338 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.710281    2558 generic.go:356] "Generic (PLEG): container finished" podID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerID="76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab" exitCode=0
Jul 22 18:57:25.710338 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.710287    2558 generic.go:356] "Generic (PLEG): container finished" podID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerID="481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299" exitCode=0
Jul 22 18:57:25.710338 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.710294    2558 generic.go:356] "Generic (PLEG): container finished" podID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerID="7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a" exitCode=0
Jul 22 18:57:25.710338 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.710326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerDied","Data":"e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4"}
Jul 22 18:57:25.710627 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.710408    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerDied","Data":"76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab"}
Jul 22 18:57:25.710627 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.710448    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerDied","Data":"481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299"}
Jul 22 18:57:25.710627 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:25.710484    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerDied","Data":"7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a"}
Jul 22 18:57:26.675070 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.675046    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:26.718003 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.717967    2558 generic.go:356] "Generic (PLEG): container finished" podID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerID="835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51" exitCode=0
Jul 22 18:57:26.718349 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.718333    2558 generic.go:356] "Generic (PLEG): container finished" podID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerID="eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a" exitCode=0
Jul 22 18:57:26.718512 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.718297    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerDied","Data":"835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51"}
Jul 22 18:57:26.719228 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.718277    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:26.719492 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.719351    2558 scope.go:117] "RemoveContainer" containerID="e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4"
Jul 22 18:57:26.720674 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.720578    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerDied","Data":"eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a"}
Jul 22 18:57:26.721022 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.720667    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"86feaef0-8bc9-4342-abc2-77fcee65ffdd","Type":"ContainerDied","Data":"50a00aa6bc583b3ecb10b50714896aaf6fa96efe5b2dd525bdf2ce015b59e532"}
Jul 22 18:57:26.727765 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.727724    2558 scope.go:117] "RemoveContainer" containerID="835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51"
Jul 22 18:57:26.735373 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.735355    2558 scope.go:117] "RemoveContainer" containerID="76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab"
Jul 22 18:57:26.742728 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.742714    2558 scope.go:117] "RemoveContainer" containerID="eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a"
Jul 22 18:57:26.749983 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.749967    2558 scope.go:117] "RemoveContainer" containerID="481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299"
Jul 22 18:57:26.756522 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.756503    2558 scope.go:117] "RemoveContainer" containerID="7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a"
Jul 22 18:57:26.762882 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.762869    2558 scope.go:117] "RemoveContainer" containerID="d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace"
Jul 22 18:57:26.769076 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.769058    2558 scope.go:117] "RemoveContainer" containerID="e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4"
Jul 22 18:57:26.769326 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:57:26.769308    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4\": container with ID starting with e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4 not found: ID does not exist" containerID="e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4"
Jul 22 18:57:26.769376 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.769333    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4"} err="failed to get container status \"e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4\": rpc error: code = NotFound desc = could not find container \"e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4\": container with ID starting with e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4 not found: ID does not exist"
Jul 22 18:57:26.769376 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.769351    2558 scope.go:117] "RemoveContainer" containerID="835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51"
Jul 22 18:57:26.769579 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:57:26.769563    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51\": container with ID starting with 835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51 not found: ID does not exist" containerID="835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51"
Jul 22 18:57:26.769620 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.769581    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51"} err="failed to get container status \"835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51\": rpc error: code = NotFound desc = could not find container \"835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51\": container with ID starting with 835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51 not found: ID does not exist"
Jul 22 18:57:26.769620 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.769592    2558 scope.go:117] "RemoveContainer" containerID="76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab"
Jul 22 18:57:26.769786 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:57:26.769767    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab\": container with ID starting with 76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab not found: ID does not exist" containerID="76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab"
Jul 22 18:57:26.769879 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.769789    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab"} err="failed to get container status \"76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab\": rpc error: code = NotFound desc = could not find container \"76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab\": container with ID starting with 76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab not found: ID does not exist"
Jul 22 18:57:26.769879 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.769802    2558 scope.go:117] "RemoveContainer" containerID="eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a"
Jul 22 18:57:26.770030 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:57:26.770012    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a\": container with ID starting with eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a not found: ID does not exist" containerID="eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a"
Jul 22 18:57:26.770069 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770034    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a"} err="failed to get container status \"eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a\": rpc error: code = NotFound desc = could not find container \"eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a\": container with ID starting with eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a not found: ID does not exist"
Jul 22 18:57:26.770069 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770047    2558 scope.go:117] "RemoveContainer" containerID="481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299"
Jul 22 18:57:26.770262 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:57:26.770248    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299\": container with ID starting with 481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299 not found: ID does not exist" containerID="481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299"
Jul 22 18:57:26.770303 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770266    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299"} err="failed to get container status \"481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299\": rpc error: code = NotFound desc = could not find container \"481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299\": container with ID starting with 481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299 not found: ID does not exist"
Jul 22 18:57:26.770303 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770277    2558 scope.go:117] "RemoveContainer" containerID="7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a"
Jul 22 18:57:26.770474 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:57:26.770459    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a\": container with ID starting with 7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a not found: ID does not exist" containerID="7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a"
Jul 22 18:57:26.770517 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770476    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a"} err="failed to get container status \"7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a\": rpc error: code = NotFound desc = could not find container \"7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a\": container with ID starting with 7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a not found: ID does not exist"
Jul 22 18:57:26.770517 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770487    2558 scope.go:117] "RemoveContainer" containerID="d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace"
Jul 22 18:57:26.770679 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:57:26.770665    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace\": container with ID starting with d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace not found: ID does not exist" containerID="d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace"
Jul 22 18:57:26.770720 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770682    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace"} err="failed to get container status \"d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace\": rpc error: code = NotFound desc = could not find container \"d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace\": container with ID starting with d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace not found: ID does not exist"
Jul 22 18:57:26.770720 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770693    2558 scope.go:117] "RemoveContainer" containerID="e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4"
Jul 22 18:57:26.770949 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770933    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4"} err="failed to get container status \"e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4\": rpc error: code = NotFound desc = could not find container \"e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4\": container with ID starting with e11958cf48e8f8d014500fd0b4ee19c09cd30c545db4e00158fa1e747f7d76e4 not found: ID does not exist"
Jul 22 18:57:26.770987 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.770949    2558 scope.go:117] "RemoveContainer" containerID="835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51"
Jul 22 18:57:26.771162 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771145    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51"} err="failed to get container status \"835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51\": rpc error: code = NotFound desc = could not find container \"835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51\": container with ID starting with 835dd9f453a116e7776846a7c59bd33889f6cf0654955c051f5a3f5ed7063e51 not found: ID does not exist"
Jul 22 18:57:26.771208 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771163    2558 scope.go:117] "RemoveContainer" containerID="76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab"
Jul 22 18:57:26.771357 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771342    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab"} err="failed to get container status \"76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab\": rpc error: code = NotFound desc = could not find container \"76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab\": container with ID starting with 76f1f9bf6e2d27b95b31dbb53013ea26a95ac9a28558d9038e9d80da50666aab not found: ID does not exist"
Jul 22 18:57:26.771395 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771357    2558 scope.go:117] "RemoveContainer" containerID="eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a"
Jul 22 18:57:26.771548 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771532    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a"} err="failed to get container status \"eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a\": rpc error: code = NotFound desc = could not find container \"eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a\": container with ID starting with eb962279f738488be9da4932cb50417cc8f69c437db82f3f59ecb33786a4022a not found: ID does not exist"
Jul 22 18:57:26.771589 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771548    2558 scope.go:117] "RemoveContainer" containerID="481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299"
Jul 22 18:57:26.771726 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771712    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299"} err="failed to get container status \"481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299\": rpc error: code = NotFound desc = could not find container \"481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299\": container with ID starting with 481cfbdeaa94a5e78eb7d7e26083e4ffc6dc1602454e98bc97809dd0e9668299 not found: ID does not exist"
Jul 22 18:57:26.771726 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771726    2558 scope.go:117] "RemoveContainer" containerID="7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a"
Jul 22 18:57:26.771944 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771928    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a"} err="failed to get container status \"7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a\": rpc error: code = NotFound desc = could not find container \"7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a\": container with ID starting with 7ee9450f35d10094fde539f23b823f119ea90a3798cb37649875139151e2ca8a not found: ID does not exist"
Jul 22 18:57:26.771981 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.771944    2558 scope.go:117] "RemoveContainer" containerID="d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace"
Jul 22 18:57:26.772137 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.772121    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace"} err="failed to get container status \"d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace\": rpc error: code = NotFound desc = could not find container \"d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace\": container with ID starting with d56db38a849f50f76f3c26bb5b8db9e7532f76a1c23bae4e564bd0b1bdd03ace not found: ID does not exist"
Jul 22 18:57:26.838430 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838394    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-web\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838430 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838432    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-cluster-tls-config\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838596 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838457    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-tls-assets\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838596 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838476    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-main-tls\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838596 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838496    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-metrics-client-ca\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838596 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838525    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-out\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838596 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838559    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-web-config\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838927 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838606    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sg9r\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-kube-api-access-2sg9r\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838927 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838635    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838927 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838677    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-volume\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838927 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838709    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-trusted-ca-bundle\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838927 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838733    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.838927 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.838784    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-main-db\") pod \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\" (UID: \"86feaef0-8bc9-4342-abc2-77fcee65ffdd\") "
Jul 22 18:57:26.839216 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.839097    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 18:57:26.839306 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.839282    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 18:57:26.839563 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.839536    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 18:57:26.841166 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.841145    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:57:26.842180 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.842038    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-kube-api-access-2sg9r" (OuterVolumeSpecName: "kube-api-access-2sg9r") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "kube-api-access-2sg9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:57:26.842180 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.842070    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-out" (OuterVolumeSpecName: "config-out") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 18:57:26.842180 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.842146    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 18:57:26.842180 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.842156    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:57:26.842365 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.842177    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:57:26.842530 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.842510    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-volume" (OuterVolumeSpecName: "config-volume") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:57:26.843341 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.843322    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:57:26.845820 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.845744    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:57:26.851848 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.851830    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-web-config" (OuterVolumeSpecName: "web-config") pod "86feaef0-8bc9-4342-abc2-77fcee65ffdd" (UID: "86feaef0-8bc9-4342-abc2-77fcee65ffdd"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 18:57:26.939779 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939742    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sg9r\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-kube-api-access-2sg9r\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939779 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939778    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939788    2558 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-volume\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939797    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939807    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939816    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-alertmanager-main-db\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939824    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939832    2558 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-cluster-tls-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939840    2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/86feaef0-8bc9-4342-abc2-77fcee65ffdd-tls-assets\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939849    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-secret-alertmanager-main-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939857    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86feaef0-8bc9-4342-abc2-77fcee65ffdd-metrics-client-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939865    2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/86feaef0-8bc9-4342-abc2-77fcee65ffdd-config-out\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:26.939923 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:26.939873    2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/86feaef0-8bc9-4342-abc2-77fcee65ffdd-web-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 18:57:27.041918 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.041890    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 18:57:27.049380 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.047709    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 18:57:27.072105 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072080    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 18:57:27.072383 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072366    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="init-config-reloader"
Jul 22 18:57:27.072455 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072386    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="init-config-reloader"
Jul 22 18:57:27.072455 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072397    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy-metric"
Jul 22 18:57:27.072455 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072405    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy-metric"
Jul 22 18:57:27.072455 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072418    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="config-reloader"
Jul 22 18:57:27.072455 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072427    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="config-reloader"
Jul 22 18:57:27.072455 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072436    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy-web"
Jul 22 18:57:27.072455 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072444    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy-web"
Jul 22 18:57:27.072455 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072454    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="prom-label-proxy"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072462    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="prom-label-proxy"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072472    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="alertmanager"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072481    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="alertmanager"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072489    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072497    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072524    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c835e5f-9588-49c1-a80b-f47b83adb40f" containerName="registry"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072532    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c835e5f-9588-49c1-a80b-f47b83adb40f" containerName="registry"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072590    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy-metric"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072602    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy-web"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072612    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="config-reloader"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072620    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="kube-rbac-proxy"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072629    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="prom-label-proxy"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072638    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c835e5f-9588-49c1-a80b-f47b83adb40f" containerName="registry"
Jul 22 18:57:27.072862 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.072647    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" containerName="alertmanager"
Jul 22 18:57:27.077610 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.077592    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.080460 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.080433    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6hpb2\""
Jul 22 18:57:27.080460 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.080444    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 22 18:57:27.080618 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.080437    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 22 18:57:27.080618 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.080498    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 22 18:57:27.080618 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.080565    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 22 18:57:27.080732 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.080645    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 22 18:57:27.080902 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.080887    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 22 18:57:27.080942 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.080888    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 22 18:57:27.080976 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.080960    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 22 18:57:27.086510 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.086493    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 22 18:57:27.089951 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.089932    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 18:57:27.241146 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241122    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241280 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241155    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241280 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241184    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241280 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241209    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241280 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241260    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-config-volume\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241463 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/64a7f412-dee4-45d8-b395-e5432c1c45d6-config-out\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241463 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241347    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/64a7f412-dee4-45d8-b395-e5432c1c45d6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241463 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241385    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a7f412-dee4-45d8-b395-e5432c1c45d6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241463 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241413    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjzcc\" (UniqueName: \"kubernetes.io/projected/64a7f412-dee4-45d8-b395-e5432c1c45d6-kube-api-access-tjzcc\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241463 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64a7f412-dee4-45d8-b395-e5432c1c45d6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241463 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241451    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/64a7f412-dee4-45d8-b395-e5432c1c45d6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241652 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241467    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-web-config\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.241652 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.241485    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.342818 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.342744    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.342818 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.342792    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.342818 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.342815    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343019 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.342841    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-config-volume\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343019 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.342872    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/64a7f412-dee4-45d8-b395-e5432c1c45d6-config-out\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343019 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.342896    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/64a7f412-dee4-45d8-b395-e5432c1c45d6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343019 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.342939    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a7f412-dee4-45d8-b395-e5432c1c45d6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343019 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.342978    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjzcc\" (UniqueName: \"kubernetes.io/projected/64a7f412-dee4-45d8-b395-e5432c1c45d6-kube-api-access-tjzcc\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343019 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.343008    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64a7f412-dee4-45d8-b395-e5432c1c45d6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343480 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.343456    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/64a7f412-dee4-45d8-b395-e5432c1c45d6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343588 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.343506    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-web-config\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343588 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.343540    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343699 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.343585    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.343973 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.343788    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64a7f412-dee4-45d8-b395-e5432c1c45d6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.344276 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.344249    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a7f412-dee4-45d8-b395-e5432c1c45d6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.344531 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.344508    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/64a7f412-dee4-45d8-b395-e5432c1c45d6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.345393 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.345343    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/64a7f412-dee4-45d8-b395-e5432c1c45d6-config-out\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.345836 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.345810    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.345916 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.345886    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-config-volume\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.345984 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.345919    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.346173 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.346149    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.346528 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.346505    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/64a7f412-dee4-45d8-b395-e5432c1c45d6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.346607 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.346516    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-web-config\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.347507 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.347489    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.347661 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.347644    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64a7f412-dee4-45d8-b395-e5432c1c45d6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.351521 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.351502    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjzcc\" (UniqueName: \"kubernetes.io/projected/64a7f412-dee4-45d8-b395-e5432c1c45d6-kube-api-access-tjzcc\") pod \"alertmanager-main-0\" (UID: \"64a7f412-dee4-45d8-b395-e5432c1c45d6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.389036 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.389015    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 18:57:27.520357 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.520332    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 18:57:27.522350 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:57:27.522327    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a7f412_dee4_45d8_b395_e5432c1c45d6.slice/crio-ecd5a773198f371e70496557b32073f3aef7e75c336253c3add516fc8b07f337 WatchSource:0}: Error finding container ecd5a773198f371e70496557b32073f3aef7e75c336253c3add516fc8b07f337: Status 404 returned error can't find the container with id ecd5a773198f371e70496557b32073f3aef7e75c336253c3add516fc8b07f337
Jul 22 18:57:27.723053 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.723022    2558 generic.go:356] "Generic (PLEG): container finished" podID="64a7f412-dee4-45d8-b395-e5432c1c45d6" containerID="332832c5742d50fd5531564807bdffcbeccefe4fe6076d33c4f901cbbca10edf" exitCode=0
Jul 22 18:57:27.723361 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.723114    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"64a7f412-dee4-45d8-b395-e5432c1c45d6","Type":"ContainerDied","Data":"332832c5742d50fd5531564807bdffcbeccefe4fe6076d33c4f901cbbca10edf"}
Jul 22 18:57:27.723361 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:27.723178    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"64a7f412-dee4-45d8-b395-e5432c1c45d6","Type":"ContainerStarted","Data":"ecd5a773198f371e70496557b32073f3aef7e75c336253c3add516fc8b07f337"}
Jul 22 18:57:28.450839 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:28.450805    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86feaef0-8bc9-4342-abc2-77fcee65ffdd" path="/var/lib/kubelet/pods/86feaef0-8bc9-4342-abc2-77fcee65ffdd/volumes"
Jul 22 18:57:28.730982 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:28.730901    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"64a7f412-dee4-45d8-b395-e5432c1c45d6","Type":"ContainerStarted","Data":"44bae130a55f830679963361901144781627ab9c3e7a2e1eeb6c4ae14bcea01a"}
Jul 22 18:57:28.731293 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:28.730983    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"64a7f412-dee4-45d8-b395-e5432c1c45d6","Type":"ContainerStarted","Data":"296f60018d03909a117d3b27725217665404c01d399729a2b8b001e76d418568"}
Jul 22 18:57:28.731293 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:28.731023    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"64a7f412-dee4-45d8-b395-e5432c1c45d6","Type":"ContainerStarted","Data":"40e73d0cfd5a9de5cb34b0b8148a74829a682c05e987af05a89fe96df30d6a7f"}
Jul 22 18:57:28.731293 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:28.731060    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"64a7f412-dee4-45d8-b395-e5432c1c45d6","Type":"ContainerStarted","Data":"19cc084da0ef9f70fc27a4d18ea7f08715425c3c82bb317d6c369c57245a2347"}
Jul 22 18:57:28.731293 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:28.731099    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"64a7f412-dee4-45d8-b395-e5432c1c45d6","Type":"ContainerStarted","Data":"bb3bfbba745f354911465911833ff4ad91b31362fcd94633f1c795c4d0519bfb"}
Jul 22 18:57:28.731293 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:28.731134    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"64a7f412-dee4-45d8-b395-e5432c1c45d6","Type":"ContainerStarted","Data":"9ed54ce3e7e04effd0a1564a63bd392f93d58008798c9d968a45e3b0abbc2108"}
Jul 22 18:57:28.757886 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:28.757843    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.743261102 podStartE2EDuration="1.757824631s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.563529ms" isStatefulPod=false podCreationTimestamp="2026-07-22 18:57:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:28.756960825 +0000 UTC m=+398.788978382" watchObservedRunningTime="2026-07-22 18:57:28.757824631 +0000 UTC m=+398.789842209"
Jul 22 18:57:29.453083 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.453043    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-fb4898477-cmv8m"]
Jul 22 18:57:29.456814 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.456792    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.459540 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.459522    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 22 18:57:29.459654 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.459635    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 22 18:57:29.459794 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.459770    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 22 18:57:29.459909 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.459809    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8mx4c\""
Jul 22 18:57:29.459909 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.459830    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 22 18:57:29.460018 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.459915    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 22 18:57:29.466032 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.466010    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 22 18:57:29.472794 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.472770    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-fb4898477-cmv8m"]
Jul 22 18:57:29.559832 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.559790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be00b088-e3c7-4361-a6c6-3bd93357f698-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.559975 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.559892    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-telemeter-client-tls\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.559975 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.559958    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be00b088-e3c7-4361-a6c6-3bd93357f698-serving-certs-ca-bundle\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.560096 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.559990    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws2b6\" (UniqueName: \"kubernetes.io/projected/be00b088-e3c7-4361-a6c6-3bd93357f698-kube-api-access-ws2b6\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.560142 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.560124    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.560191 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.560182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-federate-client-tls\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.560273 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.560248    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-secret-telemeter-client\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.560332 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.560287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be00b088-e3c7-4361-a6c6-3bd93357f698-metrics-client-ca\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.661957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.661184    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-secret-telemeter-client\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.661957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.661227    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be00b088-e3c7-4361-a6c6-3bd93357f698-metrics-client-ca\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.661957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.661278    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be00b088-e3c7-4361-a6c6-3bd93357f698-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.661957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.661307    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-telemeter-client-tls\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.661957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.661337    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be00b088-e3c7-4361-a6c6-3bd93357f698-serving-certs-ca-bundle\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.661957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.661379    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws2b6\" (UniqueName: \"kubernetes.io/projected/be00b088-e3c7-4361-a6c6-3bd93357f698-kube-api-access-ws2b6\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.661957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.661411    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.661957 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.661446    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-federate-client-tls\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.663241 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.663191    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be00b088-e3c7-4361-a6c6-3bd93357f698-metrics-client-ca\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.663725 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.663698    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be00b088-e3c7-4361-a6c6-3bd93357f698-serving-certs-ca-bundle\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.664378 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.664348    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be00b088-e3c7-4361-a6c6-3bd93357f698-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.664937 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.664914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-telemeter-client-tls\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.665224 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.665197    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-federate-client-tls\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.665320 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.665245    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.665409 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.665391    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/be00b088-e3c7-4361-a6c6-3bd93357f698-secret-telemeter-client\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.670351 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.670331    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws2b6\" (UniqueName: \"kubernetes.io/projected/be00b088-e3c7-4361-a6c6-3bd93357f698-kube-api-access-ws2b6\") pod \"telemeter-client-fb4898477-cmv8m\" (UID: \"be00b088-e3c7-4361-a6c6-3bd93357f698\") " pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.768294 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.768038    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m"
Jul 22 18:57:29.908615 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:29.908585    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-fb4898477-cmv8m"]
Jul 22 18:57:29.911793 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:57:29.911764    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe00b088_e3c7_4361_a6c6_3bd93357f698.slice/crio-2427bb8f62d09a10a398c79b53c4bb6b2a4ce0101b2ea239e91c55eda7b098e3 WatchSource:0}: Error finding container 2427bb8f62d09a10a398c79b53c4bb6b2a4ce0101b2ea239e91c55eda7b098e3: Status 404 returned error can't find the container with id 2427bb8f62d09a10a398c79b53c4bb6b2a4ce0101b2ea239e91c55eda7b098e3
Jul 22 18:57:30.742767 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:30.742716    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m" event={"ID":"be00b088-e3c7-4361-a6c6-3bd93357f698","Type":"ContainerStarted","Data":"2427bb8f62d09a10a398c79b53c4bb6b2a4ce0101b2ea239e91c55eda7b098e3"}
Jul 22 18:57:31.748803 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:31.748710    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m" event={"ID":"be00b088-e3c7-4361-a6c6-3bd93357f698","Type":"ContainerStarted","Data":"c174e0a4883705169555309e9a4621038ede0dbd7efa824e7cf74413e233ebdd"}
Jul 22 18:57:31.748803 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:31.748792    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m" event={"ID":"be00b088-e3c7-4361-a6c6-3bd93357f698","Type":"ContainerStarted","Data":"359e11513a91d322dd99e43980552769efceb7a118302b62691cebcc7f1e2663"}
Jul 22 18:57:31.749148 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:31.748815    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m" event={"ID":"be00b088-e3c7-4361-a6c6-3bd93357f698","Type":"ContainerStarted","Data":"39d6b99daa85b74ef18adf6d6f7899f4f84242360cc6a2b1d4872ebe641fa7cc"}
Jul 22 18:57:31.774019 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:31.773965    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-fb4898477-cmv8m" podStartSLOduration=1.228649122 podStartE2EDuration="2.773949968s" totalImagesPullingTime="1.545300846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:57:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:31.77220111 +0000 UTC m=+401.804218672" watchObservedRunningTime="2026-07-22 18:57:31.773949968 +0000 UTC m=+401.805967497"
Jul 22 18:57:36.271532 ip-10-0-136-84 kubenswrapper[2558]: E0722 18:57:36.271487    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh" podUID="dfc22088-efc9-4e37-926b-a00989038020"
Jul 22 18:57:36.770334 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:36.770302    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:57:38.320282 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:38.320246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:57:38.322594 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:38.322563    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dfc22088-efc9-4e37-926b-a00989038020-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4qrhh\" (UID: \"dfc22088-efc9-4e37-926b-a00989038020\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:57:38.574279 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:38.574207    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-k994z\""
Jul 22 18:57:38.582359 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:38.582342    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"
Jul 22 18:57:38.705334 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:38.705305    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh"]
Jul 22 18:57:38.708159 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:57:38.708127    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfc22088_efc9_4e37_926b_a00989038020.slice/crio-61d17d064e38f18448827c9fb2a7ab962da56c335d478c5878689c0310f712fa WatchSource:0}: Error finding container 61d17d064e38f18448827c9fb2a7ab962da56c335d478c5878689c0310f712fa: Status 404 returned error can't find the container with id 61d17d064e38f18448827c9fb2a7ab962da56c335d478c5878689c0310f712fa
Jul 22 18:57:38.778008 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:38.777975    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh" event={"ID":"dfc22088-efc9-4e37-926b-a00989038020","Type":"ContainerStarted","Data":"61d17d064e38f18448827c9fb2a7ab962da56c335d478c5878689c0310f712fa"}
Jul 22 18:57:40.786966 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:40.786927    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh" event={"ID":"dfc22088-efc9-4e37-926b-a00989038020","Type":"ContainerStarted","Data":"6d109992b9277c4b0ba45e9bb2df4c98dd96581ce876d0a67df212c1e25a1cd6"}
Jul 22 18:57:40.806278 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:57:40.806233    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4qrhh" podStartSLOduration=407.670950249 podStartE2EDuration="6m48.806217597s" totalImagesPullingTime="1.135267348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:40.80547684 +0000 UTC m=+410.837494370" watchObservedRunningTime="2026-07-22 18:57:40.806217597 +0000 UTC m=+410.838235127"
Jul 22 18:59:44.215461 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.215384    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"]
Jul 22 18:59:44.218505 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.218489    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.221178 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.221153    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 22 18:59:44.221178 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.221173    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 22 18:59:44.221358 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.221199    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 22 18:59:44.222153 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.222135    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 22 18:59:44.222247 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.222154    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 22 18:59:44.222294 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.222243    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 22 18:59:44.222474 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.222455    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 22 18:59:44.230556 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.230539    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"]
Jul 22 18:59:44.364624 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.364593    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-ca\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.364624 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.364627    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.364841 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.364651    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zh2b\" (UniqueName: \"kubernetes.io/projected/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-kube-api-access-5zh2b\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.364841 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.364718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-hub\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.364841 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.364787    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.364841 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.364832    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.465836 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.465760    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-hub\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.465836 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.465818    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.466037 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.465849    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.466037 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.465977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-ca\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.466037 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.466004    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.466037 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.466029    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zh2b\" (UniqueName: \"kubernetes.io/projected/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-kube-api-access-5zh2b\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.466647 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.466618    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.468376 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.468353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-ca\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.468585 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.468561    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.468630 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.468580    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.468630 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.468594    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-hub\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.474454 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.474432    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zh2b\" (UniqueName: \"kubernetes.io/projected/ae1ceb18-0d8c-4a31-a6d7-f2b657311873-kube-api-access-5zh2b\") pod \"cluster-proxy-proxy-agent-8bfd4998c-rrlnx\" (UID: \"ae1ceb18-0d8c-4a31-a6d7-f2b657311873\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.536416 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.536382    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"
Jul 22 18:59:44.660386 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:44.660358    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx"]
Jul 22 18:59:44.662947 ip-10-0-136-84 kubenswrapper[2558]: W0722 18:59:44.662920    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae1ceb18_0d8c_4a31_a6d7_f2b657311873.slice/crio-cb944c7152a549fc6e4f79ac8287cf48d1f0a240ba9e00d03fabe674d93f748c WatchSource:0}: Error finding container cb944c7152a549fc6e4f79ac8287cf48d1f0a240ba9e00d03fabe674d93f748c: Status 404 returned error can't find the container with id cb944c7152a549fc6e4f79ac8287cf48d1f0a240ba9e00d03fabe674d93f748c
Jul 22 18:59:45.240412 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:45.240378    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx" event={"ID":"ae1ceb18-0d8c-4a31-a6d7-f2b657311873","Type":"ContainerStarted","Data":"cb944c7152a549fc6e4f79ac8287cf48d1f0a240ba9e00d03fabe674d93f748c"}
Jul 22 18:59:50.262394 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:50.262361    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx" event={"ID":"ae1ceb18-0d8c-4a31-a6d7-f2b657311873","Type":"ContainerStarted","Data":"fbda598153ccec43b85dacd93bcf557686f6df3b0e4fd40678a1ded64da249af"}
Jul 22 18:59:50.262394 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:50.262401    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx" event={"ID":"ae1ceb18-0d8c-4a31-a6d7-f2b657311873","Type":"ContainerStarted","Data":"97a7b7d6e0aa199d09583f359c29cc64bdb82215331ac304fc50b90c807ee335"}
Jul 22 18:59:50.262821 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:50.262414    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx" event={"ID":"ae1ceb18-0d8c-4a31-a6d7-f2b657311873","Type":"ContainerStarted","Data":"0a2156408d2b6d86b7329431745b75af1ea9e2b35eba5fda9d144dd564b82168"}
Jul 22 18:59:50.280441 ip-10-0-136-84 kubenswrapper[2558]: I0722 18:59:50.280401    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8bfd4998c-rrlnx" podStartSLOduration=1.615196802 podStartE2EDuration="6.280389548s" totalImagesPullingTime="4.665192746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:59:50.279151248 +0000 UTC m=+540.311168781" watchObservedRunningTime="2026-07-22 18:59:50.280389548 +0000 UTC m=+540.312407078"
Jul 22 19:00:43.608961 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.608929    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"]
Jul 22 19:00:43.612085 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.612070    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"
Jul 22 19:00:43.614604 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.614585    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 22 19:00:43.614732 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.614585    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-njv7g\""
Jul 22 19:00:43.615531 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.615513    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 22 19:00:43.619786 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.619765    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"]
Jul 22 19:00:43.662972 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.662951    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98325322-6573-4088-93b2-4ee90c7cbd2a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rsj4n\" (UID: \"98325322-6573-4088-93b2-4ee90c7cbd2a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"
Jul 22 19:00:43.663057 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.662984    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj47r\" (UniqueName: \"kubernetes.io/projected/98325322-6573-4088-93b2-4ee90c7cbd2a-kube-api-access-mj47r\") pod \"cert-manager-cainjector-559cdd8c87-rsj4n\" (UID: \"98325322-6573-4088-93b2-4ee90c7cbd2a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"
Jul 22 19:00:43.763687 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.763664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98325322-6573-4088-93b2-4ee90c7cbd2a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rsj4n\" (UID: \"98325322-6573-4088-93b2-4ee90c7cbd2a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"
Jul 22 19:00:43.763818 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.763711    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj47r\" (UniqueName: \"kubernetes.io/projected/98325322-6573-4088-93b2-4ee90c7cbd2a-kube-api-access-mj47r\") pod \"cert-manager-cainjector-559cdd8c87-rsj4n\" (UID: \"98325322-6573-4088-93b2-4ee90c7cbd2a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"
Jul 22 19:00:43.772838 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.772819    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98325322-6573-4088-93b2-4ee90c7cbd2a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rsj4n\" (UID: \"98325322-6573-4088-93b2-4ee90c7cbd2a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"
Jul 22 19:00:43.772978 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.772961    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj47r\" (UniqueName: \"kubernetes.io/projected/98325322-6573-4088-93b2-4ee90c7cbd2a-kube-api-access-mj47r\") pod \"cert-manager-cainjector-559cdd8c87-rsj4n\" (UID: \"98325322-6573-4088-93b2-4ee90c7cbd2a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"
Jul 22 19:00:43.920999 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:43.920929    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"
Jul 22 19:00:44.042997 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:44.042948    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n"]
Jul 22 19:00:44.045722 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:00:44.045695    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98325322_6573_4088_93b2_4ee90c7cbd2a.slice/crio-bddab26d5b4805b7e09ece305a40fcf09d51b97a93e318c4956205b595cd0a93 WatchSource:0}: Error finding container bddab26d5b4805b7e09ece305a40fcf09d51b97a93e318c4956205b595cd0a93: Status 404 returned error can't find the container with id bddab26d5b4805b7e09ece305a40fcf09d51b97a93e318c4956205b595cd0a93
Jul 22 19:00:44.465935 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:44.465903    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n" event={"ID":"98325322-6573-4088-93b2-4ee90c7cbd2a","Type":"ContainerStarted","Data":"bddab26d5b4805b7e09ece305a40fcf09d51b97a93e318c4956205b595cd0a93"}
Jul 22 19:00:48.485524 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:48.485494    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n" event={"ID":"98325322-6573-4088-93b2-4ee90c7cbd2a","Type":"ContainerStarted","Data":"333c41848803749480d9624720e2aeb2925c24510d3da1a39b96619d61ab00b9"}
Jul 22 19:00:48.502360 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:48.502317    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rsj4n" podStartSLOduration=1.160128271 podStartE2EDuration="5.502300931s" totalImagesPullingTime="4.34217266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:00:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:00:48.500689599 +0000 UTC m=+598.532707143" watchObservedRunningTime="2026-07-22 19:00:48.502300931 +0000 UTC m=+598.534318462"
Jul 22 19:00:56.187537 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.187499    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pvvnr"]
Jul 22 19:00:56.190584 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.190563    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-pvvnr"
Jul 22 19:00:56.193331 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.193307    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-xmxsn\""
Jul 22 19:00:56.200666 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.200645    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pvvnr"]
Jul 22 19:00:56.263988 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.263967    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d21fd38-0a57-4b33-b651-c0c9d23a5f6a-bound-sa-token\") pod \"cert-manager-55b4d954d5-pvvnr\" (UID: \"4d21fd38-0a57-4b33-b651-c0c9d23a5f6a\") " pod="cert-manager/cert-manager-55b4d954d5-pvvnr"
Jul 22 19:00:56.264103 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.263999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv4cj\" (UniqueName: \"kubernetes.io/projected/4d21fd38-0a57-4b33-b651-c0c9d23a5f6a-kube-api-access-bv4cj\") pod \"cert-manager-55b4d954d5-pvvnr\" (UID: \"4d21fd38-0a57-4b33-b651-c0c9d23a5f6a\") " pod="cert-manager/cert-manager-55b4d954d5-pvvnr"
Jul 22 19:00:56.364689 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.364664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d21fd38-0a57-4b33-b651-c0c9d23a5f6a-bound-sa-token\") pod \"cert-manager-55b4d954d5-pvvnr\" (UID: \"4d21fd38-0a57-4b33-b651-c0c9d23a5f6a\") " pod="cert-manager/cert-manager-55b4d954d5-pvvnr"
Jul 22 19:00:56.364807 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.364703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv4cj\" (UniqueName: \"kubernetes.io/projected/4d21fd38-0a57-4b33-b651-c0c9d23a5f6a-kube-api-access-bv4cj\") pod \"cert-manager-55b4d954d5-pvvnr\" (UID: \"4d21fd38-0a57-4b33-b651-c0c9d23a5f6a\") " pod="cert-manager/cert-manager-55b4d954d5-pvvnr"
Jul 22 19:00:56.372389 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.372363    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d21fd38-0a57-4b33-b651-c0c9d23a5f6a-bound-sa-token\") pod \"cert-manager-55b4d954d5-pvvnr\" (UID: \"4d21fd38-0a57-4b33-b651-c0c9d23a5f6a\") " pod="cert-manager/cert-manager-55b4d954d5-pvvnr"
Jul 22 19:00:56.372581 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.372560    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv4cj\" (UniqueName: \"kubernetes.io/projected/4d21fd38-0a57-4b33-b651-c0c9d23a5f6a-kube-api-access-bv4cj\") pod \"cert-manager-55b4d954d5-pvvnr\" (UID: \"4d21fd38-0a57-4b33-b651-c0c9d23a5f6a\") " pod="cert-manager/cert-manager-55b4d954d5-pvvnr"
Jul 22 19:00:56.499914 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.499860    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-pvvnr"
Jul 22 19:00:56.628181 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:56.628150    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pvvnr"]
Jul 22 19:00:56.631974 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:00:56.631947    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d21fd38_0a57_4b33_b651_c0c9d23a5f6a.slice/crio-aebd16c49af7a04f662d23e987d7f36050e80d90e7f01e840286ea8f84d8230e WatchSource:0}: Error finding container aebd16c49af7a04f662d23e987d7f36050e80d90e7f01e840286ea8f84d8230e: Status 404 returned error can't find the container with id aebd16c49af7a04f662d23e987d7f36050e80d90e7f01e840286ea8f84d8230e
Jul 22 19:00:57.520059 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:57.520027    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-pvvnr" event={"ID":"4d21fd38-0a57-4b33-b651-c0c9d23a5f6a","Type":"ContainerStarted","Data":"81264726448d368764761185915c801b9544ae22fcdc5baa5d00335da0e614ac"}
Jul 22 19:00:57.520059 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:57.520065    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-pvvnr" event={"ID":"4d21fd38-0a57-4b33-b651-c0c9d23a5f6a","Type":"ContainerStarted","Data":"aebd16c49af7a04f662d23e987d7f36050e80d90e7f01e840286ea8f84d8230e"}
Jul 22 19:00:57.536208 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:00:57.536152    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-pvvnr" podStartSLOduration=1.536133998 podStartE2EDuration="1.536133998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:00:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:00:57.535703005 +0000 UTC m=+607.567720537" watchObservedRunningTime="2026-07-22 19:00:57.536133998 +0000 UTC m=+607.568151531"
Jul 22 19:01:10.970161 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:10.970122    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"]
Jul 22 19:01:10.974780 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:10.974735    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:10.977373 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:10.977340    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 19:01:10.977584 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:10.977569    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-snk6r\""
Jul 22 19:01:10.977635 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:10.977611    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 19:01:10.977709 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:10.977693    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 19:01:10.977879 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:10.977860    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 19:01:10.997280 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:10.997256    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"]
Jul 22 19:01:11.066818 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.066788    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2a60d740-3c37-4a1f-bcdd-1c773394b164-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.066980 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.066868    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qq87\" (UniqueName: \"kubernetes.io/projected/2a60d740-3c37-4a1f-bcdd-1c773394b164-kube-api-access-2qq87\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.066980 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.066916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2a60d740-3c37-4a1f-bcdd-1c773394b164-manifests\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.066980 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.066940    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a60d740-3c37-4a1f-bcdd-1c773394b164-tmp\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.067122 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.066995    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2a60d740-3c37-4a1f-bcdd-1c773394b164-webhook-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.168035 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.168000    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2a60d740-3c37-4a1f-bcdd-1c773394b164-webhook-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.168143 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.168045    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2a60d740-3c37-4a1f-bcdd-1c773394b164-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.168143 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.168083    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qq87\" (UniqueName: \"kubernetes.io/projected/2a60d740-3c37-4a1f-bcdd-1c773394b164-kube-api-access-2qq87\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.168143 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.168109    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2a60d740-3c37-4a1f-bcdd-1c773394b164-manifests\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.168143 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.168125    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a60d740-3c37-4a1f-bcdd-1c773394b164-tmp\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.168600 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.168575    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2a60d740-3c37-4a1f-bcdd-1c773394b164-manifests\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.170363 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.170334    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a60d740-3c37-4a1f-bcdd-1c773394b164-tmp\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.170474 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.170457    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2a60d740-3c37-4a1f-bcdd-1c773394b164-webhook-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.170588 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.170573    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2a60d740-3c37-4a1f-bcdd-1c773394b164-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.176371 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.176346    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qq87\" (UniqueName: \"kubernetes.io/projected/2a60d740-3c37-4a1f-bcdd-1c773394b164-kube-api-access-2qq87\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nhlkc\" (UID: \"2a60d740-3c37-4a1f-bcdd-1c773394b164\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.286128 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.286040    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:11.424570 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.424544    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"]
Jul 22 19:01:11.427154 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:01:11.427126    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a60d740_3c37_4a1f_bcdd_1c773394b164.slice/crio-940fdc7cd867ab6094ce90c886a01299646409b5ba62cb7e412b3332e7540579 WatchSource:0}: Error finding container 940fdc7cd867ab6094ce90c886a01299646409b5ba62cb7e412b3332e7540579: Status 404 returned error can't find the container with id 940fdc7cd867ab6094ce90c886a01299646409b5ba62cb7e412b3332e7540579
Jul 22 19:01:11.429000 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.428984    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 19:01:11.572322 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:11.572263    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc" event={"ID":"2a60d740-3c37-4a1f-bcdd-1c773394b164","Type":"ContainerStarted","Data":"940fdc7cd867ab6094ce90c886a01299646409b5ba62cb7e412b3332e7540579"}
Jul 22 19:01:15.591036 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:15.591001    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc" event={"ID":"2a60d740-3c37-4a1f-bcdd-1c773394b164","Type":"ContainerStarted","Data":"90cf389cebe50d4d99f5a06127cfebc890b718b8d1358d9be46f5f43aa9e3afa"}
Jul 22 19:01:17.602209 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:17.602172    2558 generic.go:356] "Generic (PLEG): container finished" podID="2a60d740-3c37-4a1f-bcdd-1c773394b164" containerID="90cf389cebe50d4d99f5a06127cfebc890b718b8d1358d9be46f5f43aa9e3afa" exitCode=0
Jul 22 19:01:17.602585 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:17.602246    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc" event={"ID":"2a60d740-3c37-4a1f-bcdd-1c773394b164","Type":"ContainerDied","Data":"90cf389cebe50d4d99f5a06127cfebc890b718b8d1358d9be46f5f43aa9e3afa"}
Jul 22 19:01:19.612866 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:19.612830    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc" event={"ID":"2a60d740-3c37-4a1f-bcdd-1c773394b164","Type":"ContainerStarted","Data":"859c1b87a7f7b274c2f28207815790cdd23519dc17e734c050c158225493e147"}
Jul 22 19:01:19.613265 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:19.613049    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:19.635233 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:19.635182    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc" podStartSLOduration=3.606427183 podStartE2EDuration="9.635168256s" totalImagesPullingTime="5.012699895s" totalInitContainerRuntime="1.016041178s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:19.632857828 +0000 UTC m=+629.664875359" watchObservedRunningTime="2026-07-22 19:01:19.635168256 +0000 UTC m=+629.667185786"
Jul 22 19:01:30.618908 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:30.618879    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nhlkc"
Jul 22 19:01:33.120289 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.120258    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"]
Jul 22 19:01:33.125684 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.125668    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.128421 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.128396    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 19:01:33.128554 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.128441    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 19:01:33.128554 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.128451    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 19:01:33.129386 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.129367    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 19:01:33.135738 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.135713    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"]
Jul 22 19:01:33.228811 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.228786    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zps2m\" (UniqueName: \"kubernetes.io/projected/32b1a91d-debe-48a5-8ed7-33199b3f7c10-kube-api-access-zps2m\") pod \"kube-auth-proxy-86546cfdc9-lwsnj\" (UID: \"32b1a91d-debe-48a5-8ed7-33199b3f7c10\") " pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.228920 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.228820    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32b1a91d-debe-48a5-8ed7-33199b3f7c10-tmp\") pod \"kube-auth-proxy-86546cfdc9-lwsnj\" (UID: \"32b1a91d-debe-48a5-8ed7-33199b3f7c10\") " pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.228920 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.228842    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32b1a91d-debe-48a5-8ed7-33199b3f7c10-tls-certs\") pod \"kube-auth-proxy-86546cfdc9-lwsnj\" (UID: \"32b1a91d-debe-48a5-8ed7-33199b3f7c10\") " pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.329733 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.329706    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32b1a91d-debe-48a5-8ed7-33199b3f7c10-tmp\") pod \"kube-auth-proxy-86546cfdc9-lwsnj\" (UID: \"32b1a91d-debe-48a5-8ed7-33199b3f7c10\") " pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.329884 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.329737    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32b1a91d-debe-48a5-8ed7-33199b3f7c10-tls-certs\") pod \"kube-auth-proxy-86546cfdc9-lwsnj\" (UID: \"32b1a91d-debe-48a5-8ed7-33199b3f7c10\") " pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.329884 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.329808    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zps2m\" (UniqueName: \"kubernetes.io/projected/32b1a91d-debe-48a5-8ed7-33199b3f7c10-kube-api-access-zps2m\") pod \"kube-auth-proxy-86546cfdc9-lwsnj\" (UID: \"32b1a91d-debe-48a5-8ed7-33199b3f7c10\") " pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.331893 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.331872    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32b1a91d-debe-48a5-8ed7-33199b3f7c10-tmp\") pod \"kube-auth-proxy-86546cfdc9-lwsnj\" (UID: \"32b1a91d-debe-48a5-8ed7-33199b3f7c10\") " pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.332095 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.332078    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32b1a91d-debe-48a5-8ed7-33199b3f7c10-tls-certs\") pod \"kube-auth-proxy-86546cfdc9-lwsnj\" (UID: \"32b1a91d-debe-48a5-8ed7-33199b3f7c10\") " pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.343645 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.343620    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zps2m\" (UniqueName: \"kubernetes.io/projected/32b1a91d-debe-48a5-8ed7-33199b3f7c10-kube-api-access-zps2m\") pod \"kube-auth-proxy-86546cfdc9-lwsnj\" (UID: \"32b1a91d-debe-48a5-8ed7-33199b3f7c10\") " pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.435389 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.435338    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"
Jul 22 19:01:33.438887 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.438864    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-npgn5"]
Jul 22 19:01:33.441531 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.441511    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:33.443964 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.443945    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 22 19:01:33.444059 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.443986    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-gmp6l\""
Jul 22 19:01:33.453506 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.453485    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-npgn5"]
Jul 22 19:01:33.531247 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.531215    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39eb5aeb-0895-4ac6-8817-317da4b8a0d4-cert\") pod \"odh-model-controller-7998c67dc4-npgn5\" (UID: \"39eb5aeb-0895-4ac6-8817-317da4b8a0d4\") " pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:33.531370 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.531315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x9j2\" (UniqueName: \"kubernetes.io/projected/39eb5aeb-0895-4ac6-8817-317da4b8a0d4-kube-api-access-7x9j2\") pod \"odh-model-controller-7998c67dc4-npgn5\" (UID: \"39eb5aeb-0895-4ac6-8817-317da4b8a0d4\") " pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:33.565033 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.565002    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj"]
Jul 22 19:01:33.567944 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:01:33.567917    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b1a91d_debe_48a5_8ed7_33199b3f7c10.slice/crio-19fb4b52229affa6b7aa1bc6ce653fcd54933e64035f2671cfa374e8c7f250de WatchSource:0}: Error finding container 19fb4b52229affa6b7aa1bc6ce653fcd54933e64035f2671cfa374e8c7f250de: Status 404 returned error can't find the container with id 19fb4b52229affa6b7aa1bc6ce653fcd54933e64035f2671cfa374e8c7f250de
Jul 22 19:01:33.632679 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.632652    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x9j2\" (UniqueName: \"kubernetes.io/projected/39eb5aeb-0895-4ac6-8817-317da4b8a0d4-kube-api-access-7x9j2\") pod \"odh-model-controller-7998c67dc4-npgn5\" (UID: \"39eb5aeb-0895-4ac6-8817-317da4b8a0d4\") " pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:33.632815 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.632725    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39eb5aeb-0895-4ac6-8817-317da4b8a0d4-cert\") pod \"odh-model-controller-7998c67dc4-npgn5\" (UID: \"39eb5aeb-0895-4ac6-8817-317da4b8a0d4\") " pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:33.634929 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.634911    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39eb5aeb-0895-4ac6-8817-317da4b8a0d4-cert\") pod \"odh-model-controller-7998c67dc4-npgn5\" (UID: \"39eb5aeb-0895-4ac6-8817-317da4b8a0d4\") " pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:33.651621 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.651603    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x9j2\" (UniqueName: \"kubernetes.io/projected/39eb5aeb-0895-4ac6-8817-317da4b8a0d4-kube-api-access-7x9j2\") pod \"odh-model-controller-7998c67dc4-npgn5\" (UID: \"39eb5aeb-0895-4ac6-8817-317da4b8a0d4\") " pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:33.671354 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.671329    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj" event={"ID":"32b1a91d-debe-48a5-8ed7-33199b3f7c10","Type":"ContainerStarted","Data":"19fb4b52229affa6b7aa1bc6ce653fcd54933e64035f2671cfa374e8c7f250de"}
Jul 22 19:01:33.764907 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.764858    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:33.889176 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:33.889150    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-npgn5"]
Jul 22 19:01:33.892043 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:01:33.892017    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39eb5aeb_0895_4ac6_8817_317da4b8a0d4.slice/crio-9cd286cb8a165fc2fb6b596f125ff4aeb77955fb50e706fa6ebe5cc7df3d4b75 WatchSource:0}: Error finding container 9cd286cb8a165fc2fb6b596f125ff4aeb77955fb50e706fa6ebe5cc7df3d4b75: Status 404 returned error can't find the container with id 9cd286cb8a165fc2fb6b596f125ff4aeb77955fb50e706fa6ebe5cc7df3d4b75
Jul 22 19:01:34.357762 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.357725    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jxwpr"]
Jul 22 19:01:34.361123 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.361101    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:01:34.363450 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.363434    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 22 19:01:34.363518 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.363448    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jjtjv\""
Jul 22 19:01:34.371925 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.371904    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jxwpr"]
Jul 22 19:01:34.395979 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.395955    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"]
Jul 22 19:01:34.398500 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.398480    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.401723 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.401705    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 22 19:01:34.401838 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.401802    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 22 19:01:34.402084 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.402064    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 19:01:34.402190 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.402116    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 22 19:01:34.402508 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.402414    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xjlgz\""
Jul 22 19:01:34.411730 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.411701    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"]
Jul 22 19:01:34.440011 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.439991    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763af33e-9b1b-4379-a81e-359da48f65cc-tls-certs\") pod \"model-serving-api-54d89bd5fc-jxwpr\" (UID: \"763af33e-9b1b-4379-a81e-359da48f65cc\") " pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:01:34.440106 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.440023    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7gjj\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-kube-api-access-q7gjj\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.440106 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.440052    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/834ec67b-08c6-47f5-9148-c0681971e542-local-certs\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.440106 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.440077    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.440283 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.440148    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.440283 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.440219    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-istio-token\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.440283 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.440259    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-cacerts\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.440422 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.440288    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4q9h\" (UniqueName: \"kubernetes.io/projected/763af33e-9b1b-4379-a81e-359da48f65cc-kube-api-access-t4q9h\") pod \"model-serving-api-54d89bd5fc-jxwpr\" (UID: \"763af33e-9b1b-4379-a81e-359da48f65cc\") " pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:01:34.440422 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.440335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.541241 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.541157    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7gjj\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-kube-api-access-q7gjj\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.541399 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.541276    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/834ec67b-08c6-47f5-9148-c0681971e542-local-certs\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.541399 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.541307    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.541399 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.541340    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.541399 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.541389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-istio-token\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.541600 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.541439    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-cacerts\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.541600 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.541462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4q9h\" (UniqueName: \"kubernetes.io/projected/763af33e-9b1b-4379-a81e-359da48f65cc-kube-api-access-t4q9h\") pod \"model-serving-api-54d89bd5fc-jxwpr\" (UID: \"763af33e-9b1b-4379-a81e-359da48f65cc\") " pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:01:34.541600 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.541508    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.541600 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.541555    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763af33e-9b1b-4379-a81e-359da48f65cc-tls-certs\") pod \"model-serving-api-54d89bd5fc-jxwpr\" (UID: \"763af33e-9b1b-4379-a81e-359da48f65cc\") " pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:01:34.542130 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.542087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.545321 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.545278    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763af33e-9b1b-4379-a81e-359da48f65cc-tls-certs\") pod \"model-serving-api-54d89bd5fc-jxwpr\" (UID: \"763af33e-9b1b-4379-a81e-359da48f65cc\") " pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:01:34.545761 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.545708    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-cacerts\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.546282 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.546232    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.547177 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.547155    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/834ec67b-08c6-47f5-9148-c0681971e542-local-certs\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.549024 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.548984    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.553579 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.553538    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4q9h\" (UniqueName: \"kubernetes.io/projected/763af33e-9b1b-4379-a81e-359da48f65cc-kube-api-access-t4q9h\") pod \"model-serving-api-54d89bd5fc-jxwpr\" (UID: \"763af33e-9b1b-4379-a81e-359da48f65cc\") " pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:01:34.554042 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.554003    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-istio-token\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.555431 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.555404    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7gjj\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-kube-api-access-q7gjj\") pod \"istiod-openshift-gateway-7574459656-rvmv7\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.672280 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.672018    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:01:34.678597 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.678568    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-npgn5" event={"ID":"39eb5aeb-0895-4ac6-8817-317da4b8a0d4","Type":"ContainerStarted","Data":"9cd286cb8a165fc2fb6b596f125ff4aeb77955fb50e706fa6ebe5cc7df3d4b75"}
Jul 22 19:01:34.709640 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.709603    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:34.838261 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.838181    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jxwpr"]
Jul 22 19:01:34.845358 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:01:34.845327    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763af33e_9b1b_4379_a81e_359da48f65cc.slice/crio-22668f3eef4ae7c9d800924b556b904d2ecbef2d2f72c709ec554c4461706113 WatchSource:0}: Error finding container 22668f3eef4ae7c9d800924b556b904d2ecbef2d2f72c709ec554c4461706113: Status 404 returned error can't find the container with id 22668f3eef4ae7c9d800924b556b904d2ecbef2d2f72c709ec554c4461706113
Jul 22 19:01:34.893475 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:34.893405    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"]
Jul 22 19:01:34.896984 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:01:34.896955    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834ec67b_08c6_47f5_9148_c0681971e542.slice/crio-fdbb368c6d9d96cc0a6636afcb122694b4ba531cfeac22196bc064492aa012e2 WatchSource:0}: Error finding container fdbb368c6d9d96cc0a6636afcb122694b4ba531cfeac22196bc064492aa012e2: Status 404 returned error can't find the container with id fdbb368c6d9d96cc0a6636afcb122694b4ba531cfeac22196bc064492aa012e2
Jul 22 19:01:35.684692 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:35.684663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7" event={"ID":"834ec67b-08c6-47f5-9148-c0681971e542","Type":"ContainerStarted","Data":"fdbb368c6d9d96cc0a6636afcb122694b4ba531cfeac22196bc064492aa012e2"}
Jul 22 19:01:35.685942 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:35.685915    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr" event={"ID":"763af33e-9b1b-4379-a81e-359da48f65cc","Type":"ContainerStarted","Data":"22668f3eef4ae7c9d800924b556b904d2ecbef2d2f72c709ec554c4461706113"}
Jul 22 19:01:38.845910 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:38.845875    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 19:01:38.846143 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:38.845943    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 19:01:39.706383 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.706349    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj" event={"ID":"32b1a91d-debe-48a5-8ed7-33199b3f7c10","Type":"ContainerStarted","Data":"7cee02762b6f926e5cdb4fc433423b4eef04d71d7c7d1015978ec0e7c56cedf0"}
Jul 22 19:01:39.707986 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.707957    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr" event={"ID":"763af33e-9b1b-4379-a81e-359da48f65cc","Type":"ContainerStarted","Data":"27b19860a3be1b73485db1e6a009cde2ba648b4db74e4cea1a358a1651a3ce38"}
Jul 22 19:01:39.708127 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.708005    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:01:39.709591 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.709556    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-npgn5" event={"ID":"39eb5aeb-0895-4ac6-8817-317da4b8a0d4","Type":"ContainerStarted","Data":"d49e207285182417af195992d414b92421e31e8492267a0a5ff49495d766525d"}
Jul 22 19:01:39.709719 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.709701    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:39.711236 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.711215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7" event={"ID":"834ec67b-08c6-47f5-9148-c0681971e542","Type":"ContainerStarted","Data":"7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601"}
Jul 22 19:01:39.711418 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.711400    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:39.713091 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.713068    2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-rvmv7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 19:01:39.713193 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.713118    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7" podUID="834ec67b-08c6-47f5-9148-c0681971e542" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 19:01:39.724851 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.724804    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86546cfdc9-lwsnj" podStartSLOduration=1.455501669 podStartE2EDuration="6.724787449s" totalImagesPullingTime="5.26928578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:39.724216385 +0000 UTC m=+649.756233918" watchObservedRunningTime="2026-07-22 19:01:39.724787449 +0000 UTC m=+649.756804980"
Jul 22 19:01:39.788354 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.788293    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-npgn5" podStartSLOduration=1.8777912780000001 podStartE2EDuration="6.788271932s" totalImagesPullingTime="4.910480654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:39.750046308 +0000 UTC m=+649.782063839" watchObservedRunningTime="2026-07-22 19:01:39.788271932 +0000 UTC m=+649.820289464"
Jul 22 19:01:39.789958 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.789885    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7" podStartSLOduration=1.844529219 podStartE2EDuration="5.78987011s" totalImagesPullingTime="3.945340891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:39.78666423 +0000 UTC m=+649.818681790" watchObservedRunningTime="2026-07-22 19:01:39.78987011 +0000 UTC m=+649.821887640"
Jul 22 19:01:39.816380 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:39.816331    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr" podStartSLOduration=1.820283688 podStartE2EDuration="5.816313271s" totalImagesPullingTime="3.996029583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:39.814784645 +0000 UTC m=+649.846802174" watchObservedRunningTime="2026-07-22 19:01:39.816313271 +0000 UTC m=+649.848330804"
Jul 22 19:01:40.721130 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:40.721083    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:43.679395 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.679362    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"]
Jul 22 19:01:43.682307 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.682290    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:43.684635 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.684615    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 22 19:01:43.684779 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.684619    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-v28xt\""
Jul 22 19:01:43.687220 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.687200    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"]
Jul 22 19:01:43.689659 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.689404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:43.691853 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.691830    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"]
Jul 22 19:01:43.692047 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.692029    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 22 19:01:43.692047 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.692040    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-z8hjb\""
Jul 22 19:01:43.699886 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.699862    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"]
Jul 22 19:01:43.724033 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.724013    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjktg\" (UniqueName: \"kubernetes.io/projected/229e9fbf-12a8-432f-92a4-09eb4102d9b7-kube-api-access-cjktg\") pod \"kserve-controller-manager-d6f4b65b8-g2v88\" (UID: \"229e9fbf-12a8-432f-92a4-09eb4102d9b7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:43.724148 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.724065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/229e9fbf-12a8-432f-92a4-09eb4102d9b7-cert\") pod \"kserve-controller-manager-d6f4b65b8-g2v88\" (UID: \"229e9fbf-12a8-432f-92a4-09eb4102d9b7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:43.825249 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.825223    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/229e9fbf-12a8-432f-92a4-09eb4102d9b7-cert\") pod \"kserve-controller-manager-d6f4b65b8-g2v88\" (UID: \"229e9fbf-12a8-432f-92a4-09eb4102d9b7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:43.825356 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.825269    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/998d7344-e0ba-4d16-a4ba-2c100d4c89f0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4ccgg\" (UID: \"998d7344-e0ba-4d16-a4ba-2c100d4c89f0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:43.825356 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.825295    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjktg\" (UniqueName: \"kubernetes.io/projected/229e9fbf-12a8-432f-92a4-09eb4102d9b7-kube-api-access-cjktg\") pod \"kserve-controller-manager-d6f4b65b8-g2v88\" (UID: \"229e9fbf-12a8-432f-92a4-09eb4102d9b7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:43.825356 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.825334    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gs75\" (UniqueName: \"kubernetes.io/projected/998d7344-e0ba-4d16-a4ba-2c100d4c89f0-kube-api-access-4gs75\") pod \"llmisvc-controller-manager-777c6bd7d5-4ccgg\" (UID: \"998d7344-e0ba-4d16-a4ba-2c100d4c89f0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:43.825514 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:01:43.825386    2558 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 22 19:01:43.825514 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:01:43.825449    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/229e9fbf-12a8-432f-92a4-09eb4102d9b7-cert podName:229e9fbf-12a8-432f-92a4-09eb4102d9b7 nodeName:}" failed. No retries permitted until 2026-07-22 19:01:44.325430507 +0000 UTC m=+654.357448020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/229e9fbf-12a8-432f-92a4-09eb4102d9b7-cert") pod "kserve-controller-manager-d6f4b65b8-g2v88" (UID: "229e9fbf-12a8-432f-92a4-09eb4102d9b7") : secret "kserve-webhook-server-cert" not found
Jul 22 19:01:43.844565 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.844540    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjktg\" (UniqueName: \"kubernetes.io/projected/229e9fbf-12a8-432f-92a4-09eb4102d9b7-kube-api-access-cjktg\") pod \"kserve-controller-manager-d6f4b65b8-g2v88\" (UID: \"229e9fbf-12a8-432f-92a4-09eb4102d9b7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:43.926198 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.926175    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/998d7344-e0ba-4d16-a4ba-2c100d4c89f0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4ccgg\" (UID: \"998d7344-e0ba-4d16-a4ba-2c100d4c89f0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:43.926307 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.926212    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gs75\" (UniqueName: \"kubernetes.io/projected/998d7344-e0ba-4d16-a4ba-2c100d4c89f0-kube-api-access-4gs75\") pod \"llmisvc-controller-manager-777c6bd7d5-4ccgg\" (UID: \"998d7344-e0ba-4d16-a4ba-2c100d4c89f0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:43.926374 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:01:43.926330    2558 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 22 19:01:43.926435 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:01:43.926423    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998d7344-e0ba-4d16-a4ba-2c100d4c89f0-cert podName:998d7344-e0ba-4d16-a4ba-2c100d4c89f0 nodeName:}" failed. No retries permitted until 2026-07-22 19:01:44.42640401 +0000 UTC m=+654.458421539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/998d7344-e0ba-4d16-a4ba-2c100d4c89f0-cert") pod "llmisvc-controller-manager-777c6bd7d5-4ccgg" (UID: "998d7344-e0ba-4d16-a4ba-2c100d4c89f0") : secret "llmisvc-webhook-server-cert" not found
Jul 22 19:01:43.954447 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:43.954389    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gs75\" (UniqueName: \"kubernetes.io/projected/998d7344-e0ba-4d16-a4ba-2c100d4c89f0-kube-api-access-4gs75\") pod \"llmisvc-controller-manager-777c6bd7d5-4ccgg\" (UID: \"998d7344-e0ba-4d16-a4ba-2c100d4c89f0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:44.329160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:44.329078    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/229e9fbf-12a8-432f-92a4-09eb4102d9b7-cert\") pod \"kserve-controller-manager-d6f4b65b8-g2v88\" (UID: \"229e9fbf-12a8-432f-92a4-09eb4102d9b7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:44.331416 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:44.331390    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/229e9fbf-12a8-432f-92a4-09eb4102d9b7-cert\") pod \"kserve-controller-manager-d6f4b65b8-g2v88\" (UID: \"229e9fbf-12a8-432f-92a4-09eb4102d9b7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:44.430265 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:44.430240    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/998d7344-e0ba-4d16-a4ba-2c100d4c89f0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4ccgg\" (UID: \"998d7344-e0ba-4d16-a4ba-2c100d4c89f0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:44.432460 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:44.432438    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/998d7344-e0ba-4d16-a4ba-2c100d4c89f0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4ccgg\" (UID: \"998d7344-e0ba-4d16-a4ba-2c100d4c89f0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:44.593984 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:44.593922    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:44.602463 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:44.602433    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:44.737309 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:44.737287    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"]
Jul 22 19:01:44.740002 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:01:44.739972    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod229e9fbf_12a8_432f_92a4_09eb4102d9b7.slice/crio-b43736ede4943b0e52e48074825989c1b78420b651e6357486a6e392c4d41add WatchSource:0}: Error finding container b43736ede4943b0e52e48074825989c1b78420b651e6357486a6e392c4d41add: Status 404 returned error can't find the container with id b43736ede4943b0e52e48074825989c1b78420b651e6357486a6e392c4d41add
Jul 22 19:01:44.752001 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:44.751981    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"]
Jul 22 19:01:44.753616 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:01:44.753593    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod998d7344_e0ba_4d16_a4ba_2c100d4c89f0.slice/crio-01093260469face5b01dacc5e192758f47093a820126abdf6ed369579e876396 WatchSource:0}: Error finding container 01093260469face5b01dacc5e192758f47093a820126abdf6ed369579e876396: Status 404 returned error can't find the container with id 01093260469face5b01dacc5e192758f47093a820126abdf6ed369579e876396
Jul 22 19:01:45.736610 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:45.736565    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg" event={"ID":"998d7344-e0ba-4d16-a4ba-2c100d4c89f0","Type":"ContainerStarted","Data":"01093260469face5b01dacc5e192758f47093a820126abdf6ed369579e876396"}
Jul 22 19:01:45.737688 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:45.737667    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88" event={"ID":"229e9fbf-12a8-432f-92a4-09eb4102d9b7","Type":"ContainerStarted","Data":"b43736ede4943b0e52e48074825989c1b78420b651e6357486a6e392c4d41add"}
Jul 22 19:01:47.222263 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.222229    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"]
Jul 22 19:01:47.222807 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.222462    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7" podUID="834ec67b-08c6-47f5-9148-c0681971e542" containerName="discovery" containerID="cri-o://7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601" gracePeriod=30
Jul 22 19:01:47.502113 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.502089    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:47.558351 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.558323    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/834ec67b-08c6-47f5-9148-c0681971e542-local-certs\") pod \"834ec67b-08c6-47f5-9148-c0681971e542\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") "
Jul 22 19:01:47.558500 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.558363    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-istio-token\") pod \"834ec67b-08c6-47f5-9148-c0681971e542\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") "
Jul 22 19:01:47.558500 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.558387    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-cacerts\") pod \"834ec67b-08c6-47f5-9148-c0681971e542\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") "
Jul 22 19:01:47.558500 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.558433    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-ca-configmap\") pod \"834ec67b-08c6-47f5-9148-c0681971e542\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") "
Jul 22 19:01:47.558718 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.558510    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-dns-cert\") pod \"834ec67b-08c6-47f5-9148-c0681971e542\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") "
Jul 22 19:01:47.558718 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.558562    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7gjj\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-kube-api-access-q7gjj\") pod \"834ec67b-08c6-47f5-9148-c0681971e542\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") "
Jul 22 19:01:47.558718 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.558592    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-kubeconfig\") pod \"834ec67b-08c6-47f5-9148-c0681971e542\" (UID: \"834ec67b-08c6-47f5-9148-c0681971e542\") "
Jul 22 19:01:47.559295 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.559033    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "834ec67b-08c6-47f5-9148-c0681971e542" (UID: "834ec67b-08c6-47f5-9148-c0681971e542"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:01:47.561128 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.561098    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-istio-token" (OuterVolumeSpecName: "istio-token") pod "834ec67b-08c6-47f5-9148-c0681971e542" (UID: "834ec67b-08c6-47f5-9148-c0681971e542"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:01:47.561249 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.561136    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "834ec67b-08c6-47f5-9148-c0681971e542" (UID: "834ec67b-08c6-47f5-9148-c0681971e542"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:01:47.561498 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.561470    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "834ec67b-08c6-47f5-9148-c0681971e542" (UID: "834ec67b-08c6-47f5-9148-c0681971e542"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:01:47.561498 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.561476    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834ec67b-08c6-47f5-9148-c0681971e542-local-certs" (OuterVolumeSpecName: "local-certs") pod "834ec67b-08c6-47f5-9148-c0681971e542" (UID: "834ec67b-08c6-47f5-9148-c0681971e542"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:01:47.561653 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.561495    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-cacerts" (OuterVolumeSpecName: "cacerts") pod "834ec67b-08c6-47f5-9148-c0681971e542" (UID: "834ec67b-08c6-47f5-9148-c0681971e542"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:01:47.561826 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.561799    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-kube-api-access-q7gjj" (OuterVolumeSpecName: "kube-api-access-q7gjj") pod "834ec67b-08c6-47f5-9148-c0681971e542" (UID: "834ec67b-08c6-47f5-9148-c0681971e542"). InnerVolumeSpecName "kube-api-access-q7gjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:01:47.659912 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.659880    2558 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/834ec67b-08c6-47f5-9148-c0681971e542-local-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:01:47.659912 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.659916    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-istio-token\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:01:47.660082 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.659930    2558 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-cacerts\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:01:47.660082 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.659944    2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-ca-configmap\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:01:47.660082 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.659958    2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-csr-dns-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:01:47.660082 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.659971    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7gjj\" (UniqueName: \"kubernetes.io/projected/834ec67b-08c6-47f5-9148-c0681971e542-kube-api-access-q7gjj\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:01:47.660082 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.659984    2558 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/834ec67b-08c6-47f5-9148-c0681971e542-istio-kubeconfig\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:01:47.751469 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.751370    2558 generic.go:356] "Generic (PLEG): container finished" podID="834ec67b-08c6-47f5-9148-c0681971e542" containerID="7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601" exitCode=0
Jul 22 19:01:47.751469 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.751436    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"
Jul 22 19:01:47.751469 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.751456    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7" event={"ID":"834ec67b-08c6-47f5-9148-c0681971e542","Type":"ContainerDied","Data":"7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601"}
Jul 22 19:01:47.751734 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.751504    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7" event={"ID":"834ec67b-08c6-47f5-9148-c0681971e542","Type":"ContainerDied","Data":"fdbb368c6d9d96cc0a6636afcb122694b4ba531cfeac22196bc064492aa012e2"}
Jul 22 19:01:47.751734 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.751530    2558 scope.go:117] "RemoveContainer" containerID="7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601"
Jul 22 19:01:47.762378 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.762355    2558 scope.go:117] "RemoveContainer" containerID="7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601"
Jul 22 19:01:47.762718 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:01:47.762693    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601\": container with ID starting with 7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601 not found: ID does not exist" containerID="7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601"
Jul 22 19:01:47.762858 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.762730    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601"} err="failed to get container status \"7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601\": rpc error: code = NotFound desc = could not find container \"7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601\": container with ID starting with 7e896357c7a690a1c05e773b743a7a0b4b2a086ff69a96aae361e8c77292a601 not found: ID does not exist"
Jul 22 19:01:47.777399 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.777377    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"]
Jul 22 19:01:47.781837 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:47.781814    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rvmv7"]
Jul 22 19:01:48.453467 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:48.453438    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="834ec67b-08c6-47f5-9148-c0681971e542" path="/var/lib/kubelet/pods/834ec67b-08c6-47f5-9148-c0681971e542/volumes"
Jul 22 19:01:49.763983 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:49.763942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg" event={"ID":"998d7344-e0ba-4d16-a4ba-2c100d4c89f0","Type":"ContainerStarted","Data":"c1b429e2d2c56a8b15d37d3dd234c1d5c6225e57fb02733101b6b3867ec8b80d"}
Jul 22 19:01:49.764430 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:49.764082    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:01:49.765540 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:49.765513    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88" event={"ID":"229e9fbf-12a8-432f-92a4-09eb4102d9b7","Type":"ContainerStarted","Data":"0cd437673284894ed0c85970c2ac17a80d0b16a592cec3e8942f0b26d4cd0f01"}
Jul 22 19:01:49.765660 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:49.765553    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:01:49.781493 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:49.781449    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg" podStartSLOduration=2.194370333 podStartE2EDuration="6.781435642s" totalImagesPullingTime="4.587065309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:49.779230725 +0000 UTC m=+659.811248289" watchObservedRunningTime="2026-07-22 19:01:49.781435642 +0000 UTC m=+659.813453171"
Jul 22 19:01:49.795228 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:49.795187    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88" podStartSLOduration=2.193366498 podStartE2EDuration="6.795174667s" totalImagesPullingTime="4.601808169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:49.794872197 +0000 UTC m=+659.826889728" watchObservedRunningTime="2026-07-22 19:01:49.795174667 +0000 UTC m=+659.827192197"
Jul 22 19:01:50.718094 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:50.718063    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-npgn5"
Jul 22 19:01:50.720036 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:01:50.720013    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-jxwpr"
Jul 22 19:02:20.772916 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:20.772884    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4ccgg"
Jul 22 19:02:20.775986 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:20.775966    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g2v88"
Jul 22 19:02:33.969133 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:33.969100    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-265ql"]
Jul 22 19:02:33.969767 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:33.969723    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834ec67b-08c6-47f5-9148-c0681971e542" containerName="discovery"
Jul 22 19:02:33.969767 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:33.969770    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="834ec67b-08c6-47f5-9148-c0681971e542" containerName="discovery"
Jul 22 19:02:33.969952 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:33.969895    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="834ec67b-08c6-47f5-9148-c0681971e542" containerName="discovery"
Jul 22 19:02:33.974533 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:33.974509    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-265ql"
Jul 22 19:02:33.976964 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:33.976941    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 19:02:33.977265 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:33.977244    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 19:02:33.978145 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:33.978119    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rvd6q\""
Jul 22 19:02:33.983362 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:33.983342    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-265ql"]
Jul 22 19:02:34.004487 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.004463    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b44j\" (UniqueName: \"kubernetes.io/projected/05817a2b-d63d-4d7c-9ae0-389a387649fb-kube-api-access-4b44j\") pod \"kuadrant-operator-catalog-265ql\" (UID: \"05817a2b-d63d-4d7c-9ae0-389a387649fb\") " pod="kuadrant-system/kuadrant-operator-catalog-265ql"
Jul 22 19:02:34.105387 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.105359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b44j\" (UniqueName: \"kubernetes.io/projected/05817a2b-d63d-4d7c-9ae0-389a387649fb-kube-api-access-4b44j\") pod \"kuadrant-operator-catalog-265ql\" (UID: \"05817a2b-d63d-4d7c-9ae0-389a387649fb\") " pod="kuadrant-system/kuadrant-operator-catalog-265ql"
Jul 22 19:02:34.115872 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.115852    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b44j\" (UniqueName: \"kubernetes.io/projected/05817a2b-d63d-4d7c-9ae0-389a387649fb-kube-api-access-4b44j\") pod \"kuadrant-operator-catalog-265ql\" (UID: \"05817a2b-d63d-4d7c-9ae0-389a387649fb\") " pod="kuadrant-system/kuadrant-operator-catalog-265ql"
Jul 22 19:02:34.285887 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.285817    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-265ql"
Jul 22 19:02:34.328849 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.328820    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-265ql"]
Jul 22 19:02:34.412824 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.412801    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-265ql"]
Jul 22 19:02:34.415097 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:02:34.415062    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05817a2b_d63d_4d7c_9ae0_389a387649fb.slice/crio-5b70c6a92327f01500019e5801eb8496f3281a05faf718757e43050e8290a664 WatchSource:0}: Error finding container 5b70c6a92327f01500019e5801eb8496f3281a05faf718757e43050e8290a664: Status 404 returned error can't find the container with id 5b70c6a92327f01500019e5801eb8496f3281a05faf718757e43050e8290a664
Jul 22 19:02:34.538992 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.538934    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jjwqh"]
Jul 22 19:02:34.541759 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.541730    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jjwqh"
Jul 22 19:02:34.548769 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.548722    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jjwqh"]
Jul 22 19:02:34.610159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.610140    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8wp7\" (UniqueName: \"kubernetes.io/projected/cecdec2f-ea8d-468a-a388-ab01658a99ee-kube-api-access-v8wp7\") pod \"kuadrant-operator-catalog-jjwqh\" (UID: \"cecdec2f-ea8d-468a-a388-ab01658a99ee\") " pod="kuadrant-system/kuadrant-operator-catalog-jjwqh"
Jul 22 19:02:34.710937 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.710913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8wp7\" (UniqueName: \"kubernetes.io/projected/cecdec2f-ea8d-468a-a388-ab01658a99ee-kube-api-access-v8wp7\") pod \"kuadrant-operator-catalog-jjwqh\" (UID: \"cecdec2f-ea8d-468a-a388-ab01658a99ee\") " pod="kuadrant-system/kuadrant-operator-catalog-jjwqh"
Jul 22 19:02:34.718801 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.718779    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8wp7\" (UniqueName: \"kubernetes.io/projected/cecdec2f-ea8d-468a-a388-ab01658a99ee-kube-api-access-v8wp7\") pod \"kuadrant-operator-catalog-jjwqh\" (UID: \"cecdec2f-ea8d-468a-a388-ab01658a99ee\") " pod="kuadrant-system/kuadrant-operator-catalog-jjwqh"
Jul 22 19:02:34.850881 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.850859    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jjwqh"
Jul 22 19:02:34.966543 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.966510    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-265ql" event={"ID":"05817a2b-d63d-4d7c-9ae0-389a387649fb","Type":"ContainerStarted","Data":"5b70c6a92327f01500019e5801eb8496f3281a05faf718757e43050e8290a664"}
Jul 22 19:02:34.975571 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:34.975549    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jjwqh"]
Jul 22 19:02:34.977738 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:02:34.977714    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcecdec2f_ea8d_468a_a388_ab01658a99ee.slice/crio-5ba5e932ea0e27f153d270df742d3ccc6bd75efa51bf9e7e6963eec860b2cdde WatchSource:0}: Error finding container 5ba5e932ea0e27f153d270df742d3ccc6bd75efa51bf9e7e6963eec860b2cdde: Status 404 returned error can't find the container with id 5ba5e932ea0e27f153d270df742d3ccc6bd75efa51bf9e7e6963eec860b2cdde
Jul 22 19:02:35.973323 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:35.973261    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jjwqh" event={"ID":"cecdec2f-ea8d-468a-a388-ab01658a99ee","Type":"ContainerStarted","Data":"5ba5e932ea0e27f153d270df742d3ccc6bd75efa51bf9e7e6963eec860b2cdde"}
Jul 22 19:02:36.979315 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:36.979274    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-265ql" event={"ID":"05817a2b-d63d-4d7c-9ae0-389a387649fb","Type":"ContainerStarted","Data":"92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91"}
Jul 22 19:02:36.979832 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:36.979284    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-265ql" podUID="05817a2b-d63d-4d7c-9ae0-389a387649fb" containerName="registry-server" containerID="cri-o://92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91" gracePeriod=2
Jul 22 19:02:36.980939 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:36.980907    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jjwqh" event={"ID":"cecdec2f-ea8d-468a-a388-ab01658a99ee","Type":"ContainerStarted","Data":"0cd07bf5734c1b7982dbf90fc6c1b9c1e64c57fd68931844faa92d310c701f74"}
Jul 22 19:02:36.994333 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:36.994285    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-265ql" podStartSLOduration=1.80860569 podStartE2EDuration="3.99427254s" totalImagesPullingTime="2.18566685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:02:36.993653856 +0000 UTC m=+707.025671387" watchObservedRunningTime="2026-07-22 19:02:36.99427254 +0000 UTC m=+707.026290069"
Jul 22 19:02:37.008739 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.008705    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jjwqh" podStartSLOduration=1.389361276 podStartE2EDuration="3.008694539s" totalImagesPullingTime="1.619333263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:02:37.007958002 +0000 UTC m=+707.039975542" watchObservedRunningTime="2026-07-22 19:02:37.008694539 +0000 UTC m=+707.040712068"
Jul 22 19:02:37.224874 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.224851    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-265ql"
Jul 22 19:02:37.333397 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.333337    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4b44j\" (UniqueName: \"kubernetes.io/projected/05817a2b-d63d-4d7c-9ae0-389a387649fb-kube-api-access-4b44j\") pod \"05817a2b-d63d-4d7c-9ae0-389a387649fb\" (UID: \"05817a2b-d63d-4d7c-9ae0-389a387649fb\") "
Jul 22 19:02:37.335375 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.335350    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05817a2b-d63d-4d7c-9ae0-389a387649fb-kube-api-access-4b44j" (OuterVolumeSpecName: "kube-api-access-4b44j") pod "05817a2b-d63d-4d7c-9ae0-389a387649fb" (UID: "05817a2b-d63d-4d7c-9ae0-389a387649fb"). InnerVolumeSpecName "kube-api-access-4b44j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:02:37.434570 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.434544    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b44j\" (UniqueName: \"kubernetes.io/projected/05817a2b-d63d-4d7c-9ae0-389a387649fb-kube-api-access-4b44j\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:02:37.986534 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.986456    2558 generic.go:356] "Generic (PLEG): container finished" podID="05817a2b-d63d-4d7c-9ae0-389a387649fb" containerID="92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91" exitCode=0
Jul 22 19:02:37.986534 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.986498    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-265ql"
Jul 22 19:02:37.987002 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.986537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-265ql" event={"ID":"05817a2b-d63d-4d7c-9ae0-389a387649fb","Type":"ContainerDied","Data":"92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91"}
Jul 22 19:02:37.987002 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.986583    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-265ql" event={"ID":"05817a2b-d63d-4d7c-9ae0-389a387649fb","Type":"ContainerDied","Data":"5b70c6a92327f01500019e5801eb8496f3281a05faf718757e43050e8290a664"}
Jul 22 19:02:37.987002 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.986603    2558 scope.go:117] "RemoveContainer" containerID="92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91"
Jul 22 19:02:37.995580 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.995569    2558 scope.go:117] "RemoveContainer" containerID="92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91"
Jul 22 19:02:37.995834 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:02:37.995812    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91\": container with ID starting with 92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91 not found: ID does not exist" containerID="92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91"
Jul 22 19:02:37.995910 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:37.995847    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91"} err="failed to get container status \"92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91\": rpc error: code = NotFound desc = could not find container \"92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91\": container with ID starting with 92c227122ee724287cfb89a9bdf3de13ae501a93134bcdda5d42b2cb067f4c91 not found: ID does not exist"
Jul 22 19:02:38.006977 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:38.006955    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-265ql"]
Jul 22 19:02:38.010388 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:38.010366    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-265ql"]
Jul 22 19:02:38.451138 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:38.451105    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05817a2b-d63d-4d7c-9ae0-389a387649fb" path="/var/lib/kubelet/pods/05817a2b-d63d-4d7c-9ae0-389a387649fb/volumes"
Jul 22 19:02:44.851027 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:44.850952    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jjwqh"
Jul 22 19:02:44.851027 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:44.851010    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jjwqh"
Jul 22 19:02:44.873275 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:44.873252    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jjwqh"
Jul 22 19:02:45.041526 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:02:45.041502    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jjwqh"
Jul 22 19:03:06.735676 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.735639    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"]
Jul 22 19:03:06.736159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.736019    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05817a2b-d63d-4d7c-9ae0-389a387649fb" containerName="registry-server"
Jul 22 19:03:06.736159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.736034    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="05817a2b-d63d-4d7c-9ae0-389a387649fb" containerName="registry-server"
Jul 22 19:03:06.736159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.736094    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="05817a2b-d63d-4d7c-9ae0-389a387649fb" containerName="registry-server"
Jul 22 19:03:06.738544 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.738526    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"
Jul 22 19:03:06.741093 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.741072    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vh9dc\""
Jul 22 19:03:06.751836 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.751811    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"]
Jul 22 19:03:06.861889 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.861857    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5w2r\" (UniqueName: \"kubernetes.io/projected/f572820f-fda2-47df-a6fa-a8f0755fd945-kube-api-access-x5w2r\") pod \"authorino-operator-75cc8d5cb7-2l66k\" (UID: \"f572820f-fda2-47df-a6fa-a8f0755fd945\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"
Jul 22 19:03:06.962792 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.962735    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5w2r\" (UniqueName: \"kubernetes.io/projected/f572820f-fda2-47df-a6fa-a8f0755fd945-kube-api-access-x5w2r\") pod \"authorino-operator-75cc8d5cb7-2l66k\" (UID: \"f572820f-fda2-47df-a6fa-a8f0755fd945\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"
Jul 22 19:03:06.976156 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:06.976131    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5w2r\" (UniqueName: \"kubernetes.io/projected/f572820f-fda2-47df-a6fa-a8f0755fd945-kube-api-access-x5w2r\") pod \"authorino-operator-75cc8d5cb7-2l66k\" (UID: \"f572820f-fda2-47df-a6fa-a8f0755fd945\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"
Jul 22 19:03:07.048287 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:07.048225    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"
Jul 22 19:03:07.174101 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:07.174077    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"]
Jul 22 19:03:07.176567 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:03:07.176542    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf572820f_fda2_47df_a6fa_a8f0755fd945.slice/crio-61e6081d0c513e3c2e441d323596bfabb8fe391ef02c6fd9acd7d0b15c20d9c5 WatchSource:0}: Error finding container 61e6081d0c513e3c2e441d323596bfabb8fe391ef02c6fd9acd7d0b15c20d9c5: Status 404 returned error can't find the container with id 61e6081d0c513e3c2e441d323596bfabb8fe391ef02c6fd9acd7d0b15c20d9c5
Jul 22 19:03:08.130413 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:08.130369    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k" event={"ID":"f572820f-fda2-47df-a6fa-a8f0755fd945","Type":"ContainerStarted","Data":"61e6081d0c513e3c2e441d323596bfabb8fe391ef02c6fd9acd7d0b15c20d9c5"}
Jul 22 19:03:09.506022 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:09.505988    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"]
Jul 22 19:03:09.508736 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:09.508710    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:09.513973 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:09.513951    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nwm8n\""
Jul 22 19:03:09.522177 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:09.522147    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"]
Jul 22 19:03:09.587057 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:09.587025    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vprbh\" (UniqueName: \"kubernetes.io/projected/252ca927-08c8-435b-83bf-91fc397f04f2-kube-api-access-vprbh\") pod \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" (UID: \"252ca927-08c8-435b-83bf-91fc397f04f2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:09.687493 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:09.687457    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vprbh\" (UniqueName: \"kubernetes.io/projected/252ca927-08c8-435b-83bf-91fc397f04f2-kube-api-access-vprbh\") pod \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" (UID: \"252ca927-08c8-435b-83bf-91fc397f04f2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:09.706254 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:09.706232    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vprbh\" (UniqueName: \"kubernetes.io/projected/252ca927-08c8-435b-83bf-91fc397f04f2-kube-api-access-vprbh\") pod \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" (UID: \"252ca927-08c8-435b-83bf-91fc397f04f2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:09.822529 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:09.822463    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:09.949632 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:09.949605    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"]
Jul 22 19:03:09.952025 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:03:09.951996    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod252ca927_08c8_435b_83bf_91fc397f04f2.slice/crio-baf1f7a7cfc4e74a8bf7c94475a84c731bdb5e017012f0a4a411c7b1da36b95d WatchSource:0}: Error finding container baf1f7a7cfc4e74a8bf7c94475a84c731bdb5e017012f0a4a411c7b1da36b95d: Status 404 returned error can't find the container with id baf1f7a7cfc4e74a8bf7c94475a84c731bdb5e017012f0a4a411c7b1da36b95d
Jul 22 19:03:10.147181 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:10.147141    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d" event={"ID":"252ca927-08c8-435b-83bf-91fc397f04f2","Type":"ContainerStarted","Data":"baf1f7a7cfc4e74a8bf7c94475a84c731bdb5e017012f0a4a411c7b1da36b95d"}
Jul 22 19:03:10.148667 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:10.148638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k" event={"ID":"f572820f-fda2-47df-a6fa-a8f0755fd945","Type":"ContainerStarted","Data":"1ab9da2f8f226f1b30a73f989662dc6a5e4b4cab2bbfd9488479324b3aeef736"}
Jul 22 19:03:10.148894 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:10.148877    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"
Jul 22 19:03:10.166370 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:10.166326    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k" podStartSLOduration=1.753107771 podStartE2EDuration="4.166311508s" totalImagesPullingTime="2.413203737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:10.165185544 +0000 UTC m=+740.197203075" watchObservedRunningTime="2026-07-22 19:03:10.166311508 +0000 UTC m=+740.198329038"
Jul 22 19:03:12.047525 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.047491    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"]
Jul 22 19:03:12.051562 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.051542    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"
Jul 22 19:03:12.056493 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.056408    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 22 19:03:12.056493 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.056419    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-9hlnc\""
Jul 22 19:03:12.070899 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.070875    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"]
Jul 22 19:03:12.209655 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.209629    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbpql\" (UniqueName: \"kubernetes.io/projected/6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7-kube-api-access-gbpql\") pod \"dns-operator-controller-manager-864dd78c4c-qds99\" (UID: \"6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"
Jul 22 19:03:12.310523 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.310439    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbpql\" (UniqueName: \"kubernetes.io/projected/6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7-kube-api-access-gbpql\") pod \"dns-operator-controller-manager-864dd78c4c-qds99\" (UID: \"6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"
Jul 22 19:03:12.318507 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.318481    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbpql\" (UniqueName: \"kubernetes.io/projected/6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7-kube-api-access-gbpql\") pod \"dns-operator-controller-manager-864dd78c4c-qds99\" (UID: \"6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"
Jul 22 19:03:12.365017 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.364993    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"
Jul 22 19:03:12.498414 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:12.498389    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"]
Jul 22 19:03:12.501268 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:03:12.501238    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2d250d_35c8_4a03_8f6a_4d1e4f30c4f7.slice/crio-ce779cce26d902f043a67b2aef49d570e3121a4c29423e5f25f396c907fae84d WatchSource:0}: Error finding container ce779cce26d902f043a67b2aef49d570e3121a4c29423e5f25f396c907fae84d: Status 404 returned error can't find the container with id ce779cce26d902f043a67b2aef49d570e3121a4c29423e5f25f396c907fae84d
Jul 22 19:03:13.163772 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:13.163721    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99" event={"ID":"6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7","Type":"ContainerStarted","Data":"ce779cce26d902f043a67b2aef49d570e3121a4c29423e5f25f396c907fae84d"}
Jul 22 19:03:13.165304 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:13.165273    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d" event={"ID":"252ca927-08c8-435b-83bf-91fc397f04f2","Type":"ContainerStarted","Data":"321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1"}
Jul 22 19:03:13.165415 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:13.165380    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:13.185142 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:13.185100    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d" podStartSLOduration=2.014092243 podStartE2EDuration="4.185086931s" totalImagesPullingTime="2.170994688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:13.183969124 +0000 UTC m=+743.215986654" watchObservedRunningTime="2026-07-22 19:03:13.185086931 +0000 UTC m=+743.217104461"
Jul 22 19:03:15.176591 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:15.176502    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99" event={"ID":"6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7","Type":"ContainerStarted","Data":"bb761d3f26f0364de47ad95f10c8c3fbc01b064686ab951e9fe4f6c07a3ae079"}
Jul 22 19:03:15.176591 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:15.176588    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"
Jul 22 19:03:15.195036 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:15.194985    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99" podStartSLOduration=0.779239979 podStartE2EDuration="3.194970029s" totalImagesPullingTime="2.41573005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:15.19312118 +0000 UTC m=+745.225138710" watchObservedRunningTime="2026-07-22 19:03:15.194970029 +0000 UTC m=+745.226987559"
Jul 22 19:03:16.540928 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.540895    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"]
Jul 22 19:03:16.544418 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.544378    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:16.546938 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.546916    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7n4jj\""
Jul 22 19:03:16.559165 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.559142    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"]
Jul 22 19:03:16.649354 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.649327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" (UID: \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:16.649453 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.649358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6kjp\" (UniqueName: \"kubernetes.io/projected/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-kube-api-access-r6kjp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" (UID: \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:16.750235 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.750210    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6kjp\" (UniqueName: \"kubernetes.io/projected/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-kube-api-access-r6kjp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" (UID: \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:16.750324 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.750308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" (UID: \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:16.750667 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.750650    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" (UID: \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:16.761378 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.761354    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6kjp\" (UniqueName: \"kubernetes.io/projected/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-kube-api-access-r6kjp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" (UID: \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:16.853908 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.853884    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:16.991697 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:16.991673    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"]
Jul 22 19:03:16.993776 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:03:16.993737    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc976c5ec_e4d6_40b0_a6cb_fe0d78b1751e.slice/crio-a2c3d6e0875c9990cc45764cd1085ea2a67ab6d1eb818268774015e94209996e WatchSource:0}: Error finding container a2c3d6e0875c9990cc45764cd1085ea2a67ab6d1eb818268774015e94209996e: Status 404 returned error can't find the container with id a2c3d6e0875c9990cc45764cd1085ea2a67ab6d1eb818268774015e94209996e
Jul 22 19:03:17.186563 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:17.186496    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84" event={"ID":"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e","Type":"ContainerStarted","Data":"a2c3d6e0875c9990cc45764cd1085ea2a67ab6d1eb818268774015e94209996e"}
Jul 22 19:03:21.155782 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:21.155738    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2l66k"
Jul 22 19:03:24.173240 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:24.173207    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:25.228361 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:25.228323    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84" event={"ID":"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e","Type":"ContainerStarted","Data":"24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09"}
Jul 22 19:03:25.228777 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:25.228401    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:25.253337 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:25.253289    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84" podStartSLOduration=1.908491084 podStartE2EDuration="9.253274132s" totalImagesPullingTime="7.344783048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:25.250653351 +0000 UTC m=+755.282670882" watchObservedRunningTime="2026-07-22 19:03:25.253274132 +0000 UTC m=+755.285291661"
Jul 22 19:03:26.183843 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:26.183809    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qds99"
Jul 22 19:03:27.006275 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.006245    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"]
Jul 22 19:03:27.009606 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.009585    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.012081 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.012052    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 22 19:03:27.012284 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.012263    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cgr69\""
Jul 22 19:03:27.012455 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.012436    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 22 19:03:27.018356 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.018334    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"]
Jul 22 19:03:27.034837 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.034816    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e267eb7-de24-44b0-a76b-6ca12c2cc105-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vxzbv\" (UID: \"8e267eb7-de24-44b0-a76b-6ca12c2cc105\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.034923 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.034852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85blj\" (UniqueName: \"kubernetes.io/projected/8e267eb7-de24-44b0-a76b-6ca12c2cc105-kube-api-access-85blj\") pod \"kuadrant-console-plugin-66b5774985-vxzbv\" (UID: \"8e267eb7-de24-44b0-a76b-6ca12c2cc105\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.034974 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.034955    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8e267eb7-de24-44b0-a76b-6ca12c2cc105-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vxzbv\" (UID: \"8e267eb7-de24-44b0-a76b-6ca12c2cc105\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.135547 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.135525    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e267eb7-de24-44b0-a76b-6ca12c2cc105-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vxzbv\" (UID: \"8e267eb7-de24-44b0-a76b-6ca12c2cc105\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.135648 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.135560    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85blj\" (UniqueName: \"kubernetes.io/projected/8e267eb7-de24-44b0-a76b-6ca12c2cc105-kube-api-access-85blj\") pod \"kuadrant-console-plugin-66b5774985-vxzbv\" (UID: \"8e267eb7-de24-44b0-a76b-6ca12c2cc105\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.135648 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.135604    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8e267eb7-de24-44b0-a76b-6ca12c2cc105-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vxzbv\" (UID: \"8e267eb7-de24-44b0-a76b-6ca12c2cc105\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.136197 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.136173    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8e267eb7-de24-44b0-a76b-6ca12c2cc105-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-vxzbv\" (UID: \"8e267eb7-de24-44b0-a76b-6ca12c2cc105\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.137843 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.137827    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e267eb7-de24-44b0-a76b-6ca12c2cc105-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-vxzbv\" (UID: \"8e267eb7-de24-44b0-a76b-6ca12c2cc105\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.145872 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.145845    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85blj\" (UniqueName: \"kubernetes.io/projected/8e267eb7-de24-44b0-a76b-6ca12c2cc105-kube-api-access-85blj\") pod \"kuadrant-console-plugin-66b5774985-vxzbv\" (UID: \"8e267eb7-de24-44b0-a76b-6ca12c2cc105\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.319164 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.319107    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"
Jul 22 19:03:27.460300 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:27.460278    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv"]
Jul 22 19:03:27.462699 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:03:27.462669    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e267eb7_de24_44b0_a76b_6ca12c2cc105.slice/crio-60ad9f9359747c4f8500e7c1cceb9ad0871ae9685b9427000d5b3bfeb74e91f4 WatchSource:0}: Error finding container 60ad9f9359747c4f8500e7c1cceb9ad0871ae9685b9427000d5b3bfeb74e91f4: Status 404 returned error can't find the container with id 60ad9f9359747c4f8500e7c1cceb9ad0871ae9685b9427000d5b3bfeb74e91f4
Jul 22 19:03:28.242167 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:28.242133    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv" event={"ID":"8e267eb7-de24-44b0-a76b-6ca12c2cc105","Type":"ContainerStarted","Data":"60ad9f9359747c4f8500e7c1cceb9ad0871ae9685b9427000d5b3bfeb74e91f4"}
Jul 22 19:03:36.234910 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:36.234880    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:37.961833 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:37.961731    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"]
Jul 22 19:03:37.962310 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:37.961983    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" containerName="manager" containerID="cri-o://24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09" gracePeriod=2
Jul 22 19:03:37.974387 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:37.974358    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"]
Jul 22 19:03:37.996649 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:37.996622    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"]
Jul 22 19:03:37.997102 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:37.996911    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" containerName="manager" containerID="cri-o://321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1" gracePeriod=2
Jul 22 19:03:38.018842 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.018810    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"]
Jul 22 19:03:38.022157 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:38.022053    2558 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"252ca927-08c8-435b-83bf-91fc397f04f2\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-vnp7d\": pods \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:38.022684 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.022653    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"]
Jul 22 19:03:38.023213 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.023193    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" containerName="manager"
Jul 22 19:03:38.023295 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.023216    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" containerName="manager"
Jul 22 19:03:38.023295 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.023229    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" containerName="manager"
Jul 22 19:03:38.023295 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.023238    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" containerName="manager"
Jul 22 19:03:38.023411 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.023332    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" containerName="manager"
Jul 22 19:03:38.023411 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.023345    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" containerName="manager"
Jul 22 19:03:38.024472 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:38.024427    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:38.027232 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.026877    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:38.034676 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.034653    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"]
Jul 22 19:03:38.038898 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.038840    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"
Jul 22 19:03:38.053917 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.053846    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"]
Jul 22 19:03:38.067941 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:38.067892    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:38.069670 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.069648    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"]
Jul 22 19:03:38.070976 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:38.070807    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:38.073026 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:38.072885    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:38.094889 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:38.094862    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:38.137796 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.137766    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11688f13-cb30-44dd-a10c-7ee17acd244b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lfkzk\" (UID: \"11688f13-cb30-44dd-a10c-7ee17acd244b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:38.137924 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.137815    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkccx\" (UniqueName: \"kubernetes.io/projected/1f93d907-7b1a-45c3-aa08-9342e2b50153-kube-api-access-mkccx\") pod \"limitador-operator-controller-manager-859d7b78bd-677ml\" (UID: \"1f93d907-7b1a-45c3-aa08-9342e2b50153\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"
Jul 22 19:03:38.137924 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.137882    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnxnp\" (UniqueName: \"kubernetes.io/projected/11688f13-cb30-44dd-a10c-7ee17acd244b-kube-api-access-tnxnp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lfkzk\" (UID: \"11688f13-cb30-44dd-a10c-7ee17acd244b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:38.238515 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.238450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11688f13-cb30-44dd-a10c-7ee17acd244b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lfkzk\" (UID: \"11688f13-cb30-44dd-a10c-7ee17acd244b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:38.238515 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.238487    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkccx\" (UniqueName: \"kubernetes.io/projected/1f93d907-7b1a-45c3-aa08-9342e2b50153-kube-api-access-mkccx\") pod \"limitador-operator-controller-manager-859d7b78bd-677ml\" (UID: \"1f93d907-7b1a-45c3-aa08-9342e2b50153\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"
Jul 22 19:03:38.238676 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.238535    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnxnp\" (UniqueName: \"kubernetes.io/projected/11688f13-cb30-44dd-a10c-7ee17acd244b-kube-api-access-tnxnp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lfkzk\" (UID: \"11688f13-cb30-44dd-a10c-7ee17acd244b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:38.238892 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.238862    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11688f13-cb30-44dd-a10c-7ee17acd244b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lfkzk\" (UID: \"11688f13-cb30-44dd-a10c-7ee17acd244b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:38.247403 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.247376    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkccx\" (UniqueName: \"kubernetes.io/projected/1f93d907-7b1a-45c3-aa08-9342e2b50153-kube-api-access-mkccx\") pod \"limitador-operator-controller-manager-859d7b78bd-677ml\" (UID: \"1f93d907-7b1a-45c3-aa08-9342e2b50153\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"
Jul 22 19:03:38.247501 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.247376    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnxnp\" (UniqueName: \"kubernetes.io/projected/11688f13-cb30-44dd-a10c-7ee17acd244b-kube-api-access-tnxnp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lfkzk\" (UID: \"11688f13-cb30-44dd-a10c-7ee17acd244b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:38.390053 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.390022    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:38.396187 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:38.396158    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"
Jul 22 19:03:49.279104 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.279060    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:49.279548 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.279174    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"]
Jul 22 19:03:49.280377 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:03:49.280351    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11688f13_cb30_44dd_a10c_7ee17acd244b.slice/crio-deb253a5261de441f267dfb92b40d9dffefaf7760fe2011abc2bdc198d8ce987 WatchSource:0}: Error finding container deb253a5261de441f267dfb92b40d9dffefaf7760fe2011abc2bdc198d8ce987: Status 404 returned error can't find the container with id deb253a5261de441f267dfb92b40d9dffefaf7760fe2011abc2bdc198d8ce987
Jul 22 19:03:49.281549 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:49.281520    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:49.283047 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.283010    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:49.285058 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:49.285026    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:49.286971 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:49.286940    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:49.301525 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.301502    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"]
Jul 22 19:03:49.303783 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:03:49.303741    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f93d907_7b1a_45c3_aa08_9342e2b50153.slice/crio-55e4b914967b305b46165fe25b88c2da40b7ad0ed360d3bc0e6dedf24cef2cf2 WatchSource:0}: Error finding container 55e4b914967b305b46165fe25b88c2da40b7ad0ed360d3bc0e6dedf24cef2cf2: Status 404 returned error can't find the container with id 55e4b914967b305b46165fe25b88c2da40b7ad0ed360d3bc0e6dedf24cef2cf2
Jul 22 19:03:49.344514 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.344495    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6kjp\" (UniqueName: \"kubernetes.io/projected/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-kube-api-access-r6kjp\") pod \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\" (UID: \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\") "
Jul 22 19:03:49.344623 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.344525    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vprbh\" (UniqueName: \"kubernetes.io/projected/252ca927-08c8-435b-83bf-91fc397f04f2-kube-api-access-vprbh\") pod \"252ca927-08c8-435b-83bf-91fc397f04f2\" (UID: \"252ca927-08c8-435b-83bf-91fc397f04f2\") "
Jul 22 19:03:49.344623 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.344565    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-extensions-socket-volume\") pod \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\" (UID: \"c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e\") "
Jul 22 19:03:49.345212 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.345177    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" (UID: "c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:03:49.346659 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.346633    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-kube-api-access-r6kjp" (OuterVolumeSpecName: "kube-api-access-r6kjp") pod "c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" (UID: "c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e"). InnerVolumeSpecName "kube-api-access-r6kjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:03:49.346903 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.346885    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/252ca927-08c8-435b-83bf-91fc397f04f2-kube-api-access-vprbh" (OuterVolumeSpecName: "kube-api-access-vprbh") pod "252ca927-08c8-435b-83bf-91fc397f04f2" (UID: "252ca927-08c8-435b-83bf-91fc397f04f2"). InnerVolumeSpecName "kube-api-access-vprbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:03:49.367083 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.367061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml" event={"ID":"1f93d907-7b1a-45c3-aa08-9342e2b50153","Type":"ContainerStarted","Data":"7870f9e071d2712231b0da94da9a4c0c743b9f4a816331233a21aeb464fa2592"}
Jul 22 19:03:49.367175 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.367117    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"
Jul 22 19:03:49.367175 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.367142    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml" event={"ID":"1f93d907-7b1a-45c3-aa08-9342e2b50153","Type":"ContainerStarted","Data":"55e4b914967b305b46165fe25b88c2da40b7ad0ed360d3bc0e6dedf24cef2cf2"}
Jul 22 19:03:49.368628 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.368599    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv" event={"ID":"8e267eb7-de24-44b0-a76b-6ca12c2cc105","Type":"ContainerStarted","Data":"329e7508d1e4bcc9a8e7bc8e43404562f1bd38dd3d98f30ae3fbe0214c216008"}
Jul 22 19:03:49.370101 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.370079    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk" event={"ID":"11688f13-cb30-44dd-a10c-7ee17acd244b","Type":"ContainerStarted","Data":"68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352"}
Jul 22 19:03:49.370206 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.370121    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk" event={"ID":"11688f13-cb30-44dd-a10c-7ee17acd244b","Type":"ContainerStarted","Data":"deb253a5261de441f267dfb92b40d9dffefaf7760fe2011abc2bdc198d8ce987"}
Jul 22 19:03:49.370206 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.370177    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:49.371276 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.371242    2558 generic.go:356] "Generic (PLEG): container finished" podID="252ca927-08c8-435b-83bf-91fc397f04f2" containerID="321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1" exitCode=0
Jul 22 19:03:49.371276 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.371270    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:49.371396 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.371310    2558 scope.go:117] "RemoveContainer" containerID="321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1"
Jul 22 19:03:49.372540 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.372518    2558 generic.go:356] "Generic (PLEG): container finished" podID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" containerID="24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09" exitCode=0
Jul 22 19:03:49.372640 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.372557    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:49.379521 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.379502    2558 scope.go:117] "RemoveContainer" containerID="321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1"
Jul 22 19:03:49.379978 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:49.379847    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1\": container with ID starting with 321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1 not found: ID does not exist" containerID="321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1"
Jul 22 19:03:49.379978 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.379872    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1"} err="failed to get container status \"321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1\": rpc error: code = NotFound desc = could not find container \"321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1\": container with ID starting with 321eee2e357cb0870e0a4259b01f19fc3cbaf74fd7fb1c510cc6bad3cee51cb1 not found: ID does not exist"
Jul 22 19:03:49.379978 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.379889    2558 scope.go:117] "RemoveContainer" containerID="24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09"
Jul 22 19:03:49.383112 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.383075    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml" podStartSLOduration=11.383063203 podStartE2EDuration="11.383063203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:49.381215489 +0000 UTC m=+779.413233020" watchObservedRunningTime="2026-07-22 19:03:49.383063203 +0000 UTC m=+779.415080732"
Jul 22 19:03:49.383372 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:49.383339    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:49.385154 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:49.385128    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:49.388907 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.388889    2558 scope.go:117] "RemoveContainer" containerID="24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09"
Jul 22 19:03:49.389215 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:49.389187    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09\": container with ID starting with 24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09 not found: ID does not exist" containerID="24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09"
Jul 22 19:03:49.389287 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.389219    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09"} err="failed to get container status \"24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09\": rpc error: code = NotFound desc = could not find container \"24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09\": container with ID starting with 24078ac29d67381957d96df556cb4e2050225a126eefc9f2a45e2df6d6a4fc09 not found: ID does not exist"
Jul 22 19:03:49.403716 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.403670    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk" podStartSLOduration=12.403655852 podStartE2EDuration="12.403655852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:49.401892177 +0000 UTC m=+779.433909707" watchObservedRunningTime="2026-07-22 19:03:49.403655852 +0000 UTC m=+779.435673383"
Jul 22 19:03:49.403860 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:49.403687    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:49.405649 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:49.405619    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:49.426352 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.426313    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-vxzbv" podStartSLOduration=1.682871611 podStartE2EDuration="23.426301174s" totalImagesPullingTime="21.743429563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:49.42359921 +0000 UTC m=+779.455616740" watchObservedRunningTime="2026-07-22 19:03:49.426301174 +0000 UTC m=+779.458318703"
Jul 22 19:03:49.445210 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.445187    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-extensions-socket-volume\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:03:49.445322 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.445214    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6kjp\" (UniqueName: \"kubernetes.io/projected/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e-kube-api-access-r6kjp\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:03:49.445322 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:49.445229    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vprbh\" (UniqueName: \"kubernetes.io/projected/252ca927-08c8-435b-83bf-91fc397f04f2-kube-api-access-vprbh\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:03:50.449551 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:50.449484    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vnp7d\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vnp7d"
Jul 22 19:03:50.450697 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:50.450676    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="252ca927-08c8-435b-83bf-91fc397f04f2" path="/var/lib/kubelet/pods/252ca927-08c8-435b-83bf-91fc397f04f2/volumes"
Jul 22 19:03:50.451053 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:50.451035    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" path="/var/lib/kubelet/pods/c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e/volumes"
Jul 22 19:03:50.451416 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:50.451395    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8nq84\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="c976c5ec-e4d6-40b0-a6cb-fe0d78b1751e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8nq84"
Jul 22 19:03:53.915049 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:53.915015    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"]
Jul 22 19:03:53.915399 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:53.915203    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk" podUID="11688f13-cb30-44dd-a10c-7ee17acd244b" containerName="manager" containerID="cri-o://68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352" gracePeriod=10
Jul 22 19:03:54.103863 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.103833    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"]
Jul 22 19:03:54.108400 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.108378    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:03:54.122409 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.122386    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"]
Jul 22 19:03:54.169643 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.169590    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:54.181718 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.181699    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xj42z\" (UID: \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:03:54.181791 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.181734    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj5qx\" (UniqueName: \"kubernetes.io/projected/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-kube-api-access-dj5qx\") pod \"kuadrant-operator-controller-manager-86878c769f-xj42z\" (UID: \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:03:54.282344 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.282316    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnxnp\" (UniqueName: \"kubernetes.io/projected/11688f13-cb30-44dd-a10c-7ee17acd244b-kube-api-access-tnxnp\") pod \"11688f13-cb30-44dd-a10c-7ee17acd244b\" (UID: \"11688f13-cb30-44dd-a10c-7ee17acd244b\") "
Jul 22 19:03:54.282467 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.282371    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11688f13-cb30-44dd-a10c-7ee17acd244b-extensions-socket-volume\") pod \"11688f13-cb30-44dd-a10c-7ee17acd244b\" (UID: \"11688f13-cb30-44dd-a10c-7ee17acd244b\") "
Jul 22 19:03:54.282510 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.282465    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xj42z\" (UID: \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:03:54.282510 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.282492    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj5qx\" (UniqueName: \"kubernetes.io/projected/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-kube-api-access-dj5qx\") pod \"kuadrant-operator-controller-manager-86878c769f-xj42z\" (UID: \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:03:54.282824 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.282801    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11688f13-cb30-44dd-a10c-7ee17acd244b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "11688f13-cb30-44dd-a10c-7ee17acd244b" (UID: "11688f13-cb30-44dd-a10c-7ee17acd244b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:03:54.282887 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.282845    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xj42z\" (UID: \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:03:54.284403 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.284381    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11688f13-cb30-44dd-a10c-7ee17acd244b-kube-api-access-tnxnp" (OuterVolumeSpecName: "kube-api-access-tnxnp") pod "11688f13-cb30-44dd-a10c-7ee17acd244b" (UID: "11688f13-cb30-44dd-a10c-7ee17acd244b"). InnerVolumeSpecName "kube-api-access-tnxnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:03:54.291045 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.291021    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj5qx\" (UniqueName: \"kubernetes.io/projected/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-kube-api-access-dj5qx\") pod \"kuadrant-operator-controller-manager-86878c769f-xj42z\" (UID: \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:03:54.383835 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.383812    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11688f13-cb30-44dd-a10c-7ee17acd244b-extensions-socket-volume\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:03:54.383835 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.383833    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnxnp\" (UniqueName: \"kubernetes.io/projected/11688f13-cb30-44dd-a10c-7ee17acd244b-kube-api-access-tnxnp\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:03:54.397560 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.397536    2558 generic.go:356] "Generic (PLEG): container finished" podID="11688f13-cb30-44dd-a10c-7ee17acd244b" containerID="68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352" exitCode=0
Jul 22 19:03:54.397653 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.397580    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"
Jul 22 19:03:54.397653 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.397630    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk" event={"ID":"11688f13-cb30-44dd-a10c-7ee17acd244b","Type":"ContainerDied","Data":"68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352"}
Jul 22 19:03:54.397781 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.397695    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk" event={"ID":"11688f13-cb30-44dd-a10c-7ee17acd244b","Type":"ContainerDied","Data":"deb253a5261de441f267dfb92b40d9dffefaf7760fe2011abc2bdc198d8ce987"}
Jul 22 19:03:54.397781 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.397728    2558 scope.go:117] "RemoveContainer" containerID="68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352"
Jul 22 19:03:54.406402 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.406382    2558 scope.go:117] "RemoveContainer" containerID="68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352"
Jul 22 19:03:54.406642 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:03:54.406622    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352\": container with ID starting with 68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352 not found: ID does not exist" containerID="68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352"
Jul 22 19:03:54.406704 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.406650    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352"} err="failed to get container status \"68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352\": rpc error: code = NotFound desc = could not find container \"68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352\": container with ID starting with 68d5c1617a9e5461edf47aa88dc1b61222e33729c78af653ec7106ef99196352 not found: ID does not exist"
Jul 22 19:03:54.418420 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.418404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:03:54.424708 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.424688    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"]
Jul 22 19:03:54.428350 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.428331    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lfkzk"]
Jul 22 19:03:54.451124 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.451102    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11688f13-cb30-44dd-a10c-7ee17acd244b" path="/var/lib/kubelet/pods/11688f13-cb30-44dd-a10c-7ee17acd244b/volumes"
Jul 22 19:03:54.554913 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:54.554829    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"]
Jul 22 19:03:54.557658 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:03:54.557630    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7168833d_c2a1_4dcf_ad36_bedb06d97fd9.slice/crio-97aa2992348eb66602f988c4942d437d01db65eeccfcb5e4c58582f8ad0320d7 WatchSource:0}: Error finding container 97aa2992348eb66602f988c4942d437d01db65eeccfcb5e4c58582f8ad0320d7: Status 404 returned error can't find the container with id 97aa2992348eb66602f988c4942d437d01db65eeccfcb5e4c58582f8ad0320d7
Jul 22 19:03:55.404157 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:55.404116    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z" event={"ID":"7168833d-c2a1-4dcf-ad36-bedb06d97fd9","Type":"ContainerStarted","Data":"2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268"}
Jul 22 19:03:55.404576 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:55.404277    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:03:55.404576 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:55.404321    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z" event={"ID":"7168833d-c2a1-4dcf-ad36-bedb06d97fd9","Type":"ContainerStarted","Data":"97aa2992348eb66602f988c4942d437d01db65eeccfcb5e4c58582f8ad0320d7"}
Jul 22 19:03:55.428304 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:03:55.428253    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z" podStartSLOduration=1.428237614 podStartE2EDuration="1.428237614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:03:55.426684999 +0000 UTC m=+785.458702530" watchObservedRunningTime="2026-07-22 19:03:55.428237614 +0000 UTC m=+785.460255145"
Jul 22 19:04:00.379434 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:00.379402    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-677ml"
Jul 22 19:04:06.412687 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:06.412658    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:04:22.328840 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.328762    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:04:22.329289 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.329143    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11688f13-cb30-44dd-a10c-7ee17acd244b" containerName="manager"
Jul 22 19:04:22.329289 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.329154    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="11688f13-cb30-44dd-a10c-7ee17acd244b" containerName="manager"
Jul 22 19:04:22.329289 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.329215    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="11688f13-cb30-44dd-a10c-7ee17acd244b" containerName="manager"
Jul 22 19:04:22.331452 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.331432    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:04:22.333982 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.333962    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 22 19:04:22.340575 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.340553    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:04:22.374378 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.374352    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:04:22.404575 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.404548    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmjnb\" (UniqueName: \"kubernetes.io/projected/55ee36fe-f446-4197-a85e-699cdf65fe05-kube-api-access-nmjnb\") pod \"limitador-limitador-54d6d988d5-tgsbj\" (UID: \"55ee36fe-f446-4197-a85e-699cdf65fe05\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:04:22.404682 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.404632    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55ee36fe-f446-4197-a85e-699cdf65fe05-config-file\") pod \"limitador-limitador-54d6d988d5-tgsbj\" (UID: \"55ee36fe-f446-4197-a85e-699cdf65fe05\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:04:22.505267 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.505241    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmjnb\" (UniqueName: \"kubernetes.io/projected/55ee36fe-f446-4197-a85e-699cdf65fe05-kube-api-access-nmjnb\") pod \"limitador-limitador-54d6d988d5-tgsbj\" (UID: \"55ee36fe-f446-4197-a85e-699cdf65fe05\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:04:22.505387 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.505315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55ee36fe-f446-4197-a85e-699cdf65fe05-config-file\") pod \"limitador-limitador-54d6d988d5-tgsbj\" (UID: \"55ee36fe-f446-4197-a85e-699cdf65fe05\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:04:22.506104 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.506075    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/55ee36fe-f446-4197-a85e-699cdf65fe05-config-file\") pod \"limitador-limitador-54d6d988d5-tgsbj\" (UID: \"55ee36fe-f446-4197-a85e-699cdf65fe05\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:04:22.512909 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.512883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmjnb\" (UniqueName: \"kubernetes.io/projected/55ee36fe-f446-4197-a85e-699cdf65fe05-kube-api-access-nmjnb\") pod \"limitador-limitador-54d6d988d5-tgsbj\" (UID: \"55ee36fe-f446-4197-a85e-699cdf65fe05\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:04:22.642087 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.642063    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:04:22.774136 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:22.774095    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:04:22.776490 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:04:22.776465    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ee36fe_f446_4197_a85e_699cdf65fe05.slice/crio-ff93b565abaeaa5db90c9492b53286bd159d038b90565db04c26b4838ea5637f WatchSource:0}: Error finding container ff93b565abaeaa5db90c9492b53286bd159d038b90565db04c26b4838ea5637f: Status 404 returned error can't find the container with id ff93b565abaeaa5db90c9492b53286bd159d038b90565db04c26b4838ea5637f
Jul 22 19:04:23.545544 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:23.545505    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj" event={"ID":"55ee36fe-f446-4197-a85e-699cdf65fe05","Type":"ContainerStarted","Data":"ff93b565abaeaa5db90c9492b53286bd159d038b90565db04c26b4838ea5637f"}
Jul 22 19:04:25.558103 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:25.558036    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj" event={"ID":"55ee36fe-f446-4197-a85e-699cdf65fe05","Type":"ContainerStarted","Data":"6c9aa2ad3a9fb847c39ba32b80a30ac4c011989887668f6319c30a3388bb03f5"}
Jul 22 19:04:25.558425 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:25.558169    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:04:25.575520 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:25.575477    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj" podStartSLOduration=1.118932334 podStartE2EDuration="3.575463426s" totalImagesPullingTime="2.456531092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:04:25.574395103 +0000 UTC m=+815.606412633" watchObservedRunningTime="2026-07-22 19:04:25.575463426 +0000 UTC m=+815.607481018"
Jul 22 19:04:36.564301 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:04:36.564274    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"
Jul 22 19:05:34.089175 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:34.089142    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:05:57.155692 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.155656    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"]
Jul 22 19:05:57.159319 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.159304    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.162888 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.162864    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 22 19:05:57.163025 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.162886    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kcxtp\""
Jul 22 19:05:57.163025 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.162894    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 19:05:57.163025 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.162872    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 19:05:57.170354 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.170335    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"]
Jul 22 19:05:57.229574 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.229547    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.229675 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.229588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrj4n\" (UniqueName: \"kubernetes.io/projected/d8307b70-8785-47d0-ae69-f7833da861bd-kube-api-access-qrj4n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.229675 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.229619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.229675 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.229637    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.229832 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.229688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8307b70-8785-47d0-ae69-f7833da861bd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.229832 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.229770    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.330796 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.330767    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.330893 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.330815    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrj4n\" (UniqueName: \"kubernetes.io/projected/d8307b70-8785-47d0-ae69-f7833da861bd-kube-api-access-qrj4n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.330893 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.330861    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.330893 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.330887    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.331053 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.330918    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8307b70-8785-47d0-ae69-f7833da861bd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.331053 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.330966    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.331217 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.331196    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.331292 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.331270    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.331331 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.331303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.333236 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.333213    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8307b70-8785-47d0-ae69-f7833da861bd-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.333516 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.333495    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8307b70-8785-47d0-ae69-f7833da861bd-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.338301 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.338280    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrj4n\" (UniqueName: \"kubernetes.io/projected/d8307b70-8785-47d0-ae69-f7833da861bd-kube-api-access-qrj4n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf\" (UID: \"d8307b70-8785-47d0-ae69-f7833da861bd\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.470934 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.470868    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:05:57.600045 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.600019    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"]
Jul 22 19:05:57.601953 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:05:57.601916    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8307b70_8785_47d0_ae69_f7833da861bd.slice/crio-5b16f6d8f11423afd85922c9f3159b8d8b455ddca3c59797018b676ab9b3fd6e WatchSource:0}: Error finding container 5b16f6d8f11423afd85922c9f3159b8d8b455ddca3c59797018b676ab9b3fd6e: Status 404 returned error can't find the container with id 5b16f6d8f11423afd85922c9f3159b8d8b455ddca3c59797018b676ab9b3fd6e
Jul 22 19:05:57.676491 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:57.676456    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:05:58.004617 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:05:58.004576    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf" event={"ID":"d8307b70-8785-47d0-ae69-f7833da861bd","Type":"ContainerStarted","Data":"5b16f6d8f11423afd85922c9f3159b8d8b455ddca3c59797018b676ab9b3fd6e"}
Jul 22 19:06:00.017532 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:06:00.017500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf" event={"ID":"d8307b70-8785-47d0-ae69-f7833da861bd","Type":"ContainerStarted","Data":"b189d1792e71269f0613a9428c1b499504e494361d045a07778ce65947ca76b3"}
Jul 22 19:06:00.034718 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:06:00.034668    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf" podStartSLOduration=1.561052406 podStartE2EDuration="3.034654678s" totalImagesPullingTime="1.473602272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:05:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:06:00.033346504 +0000 UTC m=+910.065364029" watchObservedRunningTime="2026-07-22 19:06:00.034654678 +0000 UTC m=+910.066672238"
Jul 22 19:06:07.471301 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:06:07.471270    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:06:07.471940 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:06:07.471657    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:06:07.483900 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:06:07.483877    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:06:08.067157 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:06:08.067123    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf"
Jul 22 19:06:08.168923 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:06:08.168894    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:06:20.642112 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:06:20.642078    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:06:33.443471 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:06:33.443439    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:07:16.650777 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:07:16.650671    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:07:28.341086 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:07:28.341041    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:08:42.553793 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:08:42.553703    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:08:53.350361 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:08:53.350318    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:09:03.740608 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:09:03.740574    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:10:48.150106 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:10:48.150073    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:11:24.541835 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:11:24.541805    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:11:33.343087 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:11:33.343055    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:11:43.039157 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:11:43.039077    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:11:50.649352 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:11:50.649320    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:11:59.852813 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:11:59.852778    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:12:11.138968 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:12:11.138924    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:12:21.437310 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:12:21.437272    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:12:31.350506 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:12:31.350469    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:12:52.738354 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:12:52.738321    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:13:03.142213 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:13:03.142175    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:13:26.540849 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:13:26.540761    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:13:39.343026 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:13:39.342993    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:13:54.646944 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:13:54.646912    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:14:03.743718 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:14:03.743685    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:14:12.840887 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:14:12.840851    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:14:21.745352 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:14:21.745315    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:14:30.939878 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:14:30.939843    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:14:57.776372 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:14:57.776288    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"]
Jul 22 19:14:57.776775 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:14:57.776527    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z" podUID="7168833d-c2a1-4dcf-ad36-bedb06d97fd9" containerName="manager" containerID="cri-o://2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268" gracePeriod=10
Jul 22 19:15:00.132591 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.132570    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:15:00.143305 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.143257    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw"]
Jul 22 19:15:00.143626 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.143615    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7168833d-c2a1-4dcf-ad36-bedb06d97fd9" containerName="manager"
Jul 22 19:15:00.143667 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.143628    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7168833d-c2a1-4dcf-ad36-bedb06d97fd9" containerName="manager"
Jul 22 19:15:00.143703 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.143697    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7168833d-c2a1-4dcf-ad36-bedb06d97fd9" containerName="manager"
Jul 22 19:15:00.146510 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.146494    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw"
Jul 22 19:15:00.148840 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.148824    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\""
Jul 22 19:15:00.156636 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.156581    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw"
Jul 22 19:15:00.158723 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.158700    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw"]
Jul 22 19:15:00.240247 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.240222    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj5qx\" (UniqueName: \"kubernetes.io/projected/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-kube-api-access-dj5qx\") pod \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\" (UID: \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\") "
Jul 22 19:15:00.240388 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.240291    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-extensions-socket-volume\") pod \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\" (UID: \"7168833d-c2a1-4dcf-ad36-bedb06d97fd9\") "
Jul 22 19:15:00.240682 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.240657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7168833d-c2a1-4dcf-ad36-bedb06d97fd9" (UID: "7168833d-c2a1-4dcf-ad36-bedb06d97fd9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:15:00.242188 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.242163    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-kube-api-access-dj5qx" (OuterVolumeSpecName: "kube-api-access-dj5qx") pod "7168833d-c2a1-4dcf-ad36-bedb06d97fd9" (UID: "7168833d-c2a1-4dcf-ad36-bedb06d97fd9"). InnerVolumeSpecName "kube-api-access-dj5qx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:15:00.290317 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.290290    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw"]
Jul 22 19:15:00.292948 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:15:00.292920    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883f0f0d_c089_4811_abdf_8b06a33b2bc1.slice/crio-cfff4654e90757472cc2b68cf5c6674fdab8f9a218b2e725d0c5c448112d672d WatchSource:0}: Error finding container cfff4654e90757472cc2b68cf5c6674fdab8f9a218b2e725d0c5c448112d672d: Status 404 returned error can't find the container with id cfff4654e90757472cc2b68cf5c6674fdab8f9a218b2e725d0c5c448112d672d
Jul 22 19:15:00.294849 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.294831    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 19:15:00.340959 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.340937    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj5qx\" (UniqueName: \"kubernetes.io/projected/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-kube-api-access-dj5qx\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:15:00.340959 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.340956    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7168833d-c2a1-4dcf-ad36-bedb06d97fd9-extensions-socket-volume\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:15:00.610683 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.610652    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw" event={"ID":"883f0f0d-c089-4811-abdf-8b06a33b2bc1","Type":"ContainerStarted","Data":"cfff4654e90757472cc2b68cf5c6674fdab8f9a218b2e725d0c5c448112d672d"}
Jul 22 19:15:00.611907 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.611883    2558 generic.go:356] "Generic (PLEG): container finished" podID="7168833d-c2a1-4dcf-ad36-bedb06d97fd9" containerID="2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268" exitCode=0
Jul 22 19:15:00.612029 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.611935    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"
Jul 22 19:15:00.612093 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.611942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z" event={"ID":"7168833d-c2a1-4dcf-ad36-bedb06d97fd9","Type":"ContainerDied","Data":"2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268"}
Jul 22 19:15:00.612093 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.612068    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z" event={"ID":"7168833d-c2a1-4dcf-ad36-bedb06d97fd9","Type":"ContainerDied","Data":"97aa2992348eb66602f988c4942d437d01db65eeccfcb5e4c58582f8ad0320d7"}
Jul 22 19:15:00.612190 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.612097    2558 scope.go:117] "RemoveContainer" containerID="2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268"
Jul 22 19:15:00.624526 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.624509    2558 scope.go:117] "RemoveContainer" containerID="2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268"
Jul 22 19:15:00.624808 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:15:00.624791    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268\": container with ID starting with 2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268 not found: ID does not exist" containerID="2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268"
Jul 22 19:15:00.624879 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.624815    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268"} err="failed to get container status \"2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268\": rpc error: code = NotFound desc = could not find container \"2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268\": container with ID starting with 2dd50b1ce3c50ffbfc6eee14369cf66a47637ac94962134026a536c50fb2a268 not found: ID does not exist"
Jul 22 19:15:00.635127 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.635100    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"]
Jul 22 19:15:00.635228 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:00.635146    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xj42z"]
Jul 22 19:15:02.451656 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:02.451625    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7168833d-c2a1-4dcf-ad36-bedb06d97fd9" path="/var/lib/kubelet/pods/7168833d-c2a1-4dcf-ad36-bedb06d97fd9/volumes"
Jul 22 19:15:03.629589 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:03.629556    2558 generic.go:356] "Generic (PLEG): container finished" podID="883f0f0d-c089-4811-abdf-8b06a33b2bc1" containerID="6c5831ca3bee0e2fb326be703cbed214041fe48c4a28a4fd87d7ae2f3f7fe257" exitCode=0
Jul 22 19:15:03.629972 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:03.629625    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw" event={"ID":"883f0f0d-c089-4811-abdf-8b06a33b2bc1","Type":"ContainerDied","Data":"6c5831ca3bee0e2fb326be703cbed214041fe48c4a28a4fd87d7ae2f3f7fe257"}
Jul 22 19:15:04.771201 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:04.771170    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw"
Jul 22 19:15:05.640819 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:05.640788    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw"
Jul 22 19:15:05.640819 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:05.640815    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745795-gwjtw" event={"ID":"883f0f0d-c089-4811-abdf-8b06a33b2bc1","Type":"ContainerDied","Data":"cfff4654e90757472cc2b68cf5c6674fdab8f9a218b2e725d0c5c448112d672d"}
Jul 22 19:15:05.641064 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:05.640862    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfff4654e90757472cc2b68cf5c6674fdab8f9a218b2e725d0c5c448112d672d"
Jul 22 19:15:58.897260 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:58.897226    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"]
Jul 22 19:15:58.897719 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:58.897605    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883f0f0d-c089-4811-abdf-8b06a33b2bc1" containerName="cleanup"
Jul 22 19:15:58.897719 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:58.897618    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="883f0f0d-c089-4811-abdf-8b06a33b2bc1" containerName="cleanup"
Jul 22 19:15:58.897719 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:58.897697    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="883f0f0d-c089-4811-abdf-8b06a33b2bc1" containerName="cleanup"
Jul 22 19:15:58.901075 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:58.901047    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:15:58.905042 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:58.905022    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7n4jj\""
Jul 22 19:15:58.913968 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:58.913944    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"]
Jul 22 19:15:58.985406 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:58.985372    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61ed6eb2-badd-4022-b535-904568301275-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wjgkx\" (UID: \"61ed6eb2-badd-4022-b535-904568301275\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:15:58.985406 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:58.985407    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg54d\" (UniqueName: \"kubernetes.io/projected/61ed6eb2-badd-4022-b535-904568301275-kube-api-access-lg54d\") pod \"kuadrant-operator-controller-manager-86878c769f-wjgkx\" (UID: \"61ed6eb2-badd-4022-b535-904568301275\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:15:59.086002 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.085973    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61ed6eb2-badd-4022-b535-904568301275-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wjgkx\" (UID: \"61ed6eb2-badd-4022-b535-904568301275\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:15:59.086145 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.086009    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg54d\" (UniqueName: \"kubernetes.io/projected/61ed6eb2-badd-4022-b535-904568301275-kube-api-access-lg54d\") pod \"kuadrant-operator-controller-manager-86878c769f-wjgkx\" (UID: \"61ed6eb2-badd-4022-b535-904568301275\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:15:59.086385 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.086365    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/61ed6eb2-badd-4022-b535-904568301275-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-wjgkx\" (UID: \"61ed6eb2-badd-4022-b535-904568301275\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:15:59.094545 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.094526    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg54d\" (UniqueName: \"kubernetes.io/projected/61ed6eb2-badd-4022-b535-904568301275-kube-api-access-lg54d\") pod \"kuadrant-operator-controller-manager-86878c769f-wjgkx\" (UID: \"61ed6eb2-badd-4022-b535-904568301275\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:15:59.211985 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.211925    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:15:59.350864 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.350839    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"]
Jul 22 19:15:59.353165 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:15:59.353140    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ed6eb2_badd_4022_b535_904568301275.slice/crio-ca0ff7242acc3f0caeb9009c8a43e5e3f6e298177b1b13bb2dd1f010115e6343 WatchSource:0}: Error finding container ca0ff7242acc3f0caeb9009c8a43e5e3f6e298177b1b13bb2dd1f010115e6343: Status 404 returned error can't find the container with id ca0ff7242acc3f0caeb9009c8a43e5e3f6e298177b1b13bb2dd1f010115e6343
Jul 22 19:15:59.898730 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.898690    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx" event={"ID":"61ed6eb2-badd-4022-b535-904568301275","Type":"ContainerStarted","Data":"2a390d2bb6dc7699a2459ff6b3c261772499310ecec434eee084ca5e51e94d7a"}
Jul 22 19:15:59.899101 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.898769    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx" event={"ID":"61ed6eb2-badd-4022-b535-904568301275","Type":"ContainerStarted","Data":"ca0ff7242acc3f0caeb9009c8a43e5e3f6e298177b1b13bb2dd1f010115e6343"}
Jul 22 19:15:59.899101 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.898829    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:15:59.919368 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:15:59.919319    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx" podStartSLOduration=1.919305574 podStartE2EDuration="1.919305574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:15:59.918648788 +0000 UTC m=+1509.950666320" watchObservedRunningTime="2026-07-22 19:15:59.919305574 +0000 UTC m=+1509.951323101"
Jul 22 19:16:10.906317 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:16:10.906285    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-wjgkx"
Jul 22 19:16:20.945889 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:16:20.945813    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:16:32.136391 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:16:32.136359    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:16:42.643843 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:16:42.643806    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:16:53.242433 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:16:53.242400    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:17:04.138226 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:17:04.138197    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:17:14.836509 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:17:14.836471    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:17:24.645720 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:17:24.645684    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:17:34.442014 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:17:34.441973    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:17:45.946663 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:17:45.946586    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:17:54.949005 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:17:54.948972    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:18:05.538317 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:05.538285    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:18:30.139288 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:30.139255    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:18:37.542419 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:37.542379    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:18:42.944697 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:42.944666    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"]
Jul 22 19:18:42.948163 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:42.948143    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:42.950624 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:42.950600    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-a69122a1-kserve-self-signed-certs\""
Jul 22 19:18:42.965206 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:42.965185    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"]
Jul 22 19:18:43.021395 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.021367    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-dshm\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.021395 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.021397    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tls-certs\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.021567 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.021417    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tmp-dir\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.021611 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.021559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-model-cache\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.021611 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.021595    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj8bg\" (UniqueName: \"kubernetes.io/projected/a7214303-86d1-4caf-8dac-51a0e38b0f8c-kube-api-access-tj8bg\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.021700 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.021619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-home\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.122790 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.122739    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-model-cache\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.122923 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.122796    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj8bg\" (UniqueName: \"kubernetes.io/projected/a7214303-86d1-4caf-8dac-51a0e38b0f8c-kube-api-access-tj8bg\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.122923 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.122818    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-home\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.122923 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.122861    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-dshm\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.122923 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.122878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tls-certs\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.122923 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.122902    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tmp-dir\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.123475 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.123185    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-model-cache\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.123636 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.123612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tmp-dir\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.123854 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.123718    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-home\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.127769 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.125612    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-dshm\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.129776 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.129353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tls-certs\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.131421 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.130439    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj8bg\" (UniqueName: \"kubernetes.io/projected/a7214303-86d1-4caf-8dac-51a0e38b0f8c-kube-api-access-tj8bg\") pod \"e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") " pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.256930 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.256828    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:43.388313 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.388282    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"]
Jul 22 19:18:43.389274 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:18:43.389240    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7214303_86d1_4caf_8dac_51a0e38b0f8c.slice/crio-e1bba102359b0354b960272dfad2ad2f23ae89a55a99ba0636454a6578c64019 WatchSource:0}: Error finding container e1bba102359b0354b960272dfad2ad2f23ae89a55a99ba0636454a6578c64019: Status 404 returned error can't find the container with id e1bba102359b0354b960272dfad2ad2f23ae89a55a99ba0636454a6578c64019
Jul 22 19:18:43.694806 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.694760    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx" event={"ID":"a7214303-86d1-4caf-8dac-51a0e38b0f8c","Type":"ContainerStarted","Data":"53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9"}
Jul 22 19:18:43.694806 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.694808    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx" event={"ID":"a7214303-86d1-4caf-8dac-51a0e38b0f8c","Type":"ContainerStarted","Data":"e1bba102359b0354b960272dfad2ad2f23ae89a55a99ba0636454a6578c64019"}
Jul 22 19:18:43.712204 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:43.712157    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx" podStartSLOduration=1.7121439299999999 podStartE2EDuration="1.71214393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:18:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:18:43.709875675 +0000 UTC m=+1673.741893209" watchObservedRunningTime="2026-07-22 19:18:43.71214393 +0000 UTC m=+1673.744161457"
Jul 22 19:18:47.072464 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:47.072430    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:18:49.744263 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.744232    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"]
Jul 22 19:18:49.747899 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.747879    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.750126 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.750105    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-a69122a1-kserve-self-signed-certs\""
Jul 22 19:18:49.755556 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.755274    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"]
Jul 22 19:18:49.772688 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.772664    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-dshm\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.772803 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.772694    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-model-cache\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.772803 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.772729    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-home\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.772803 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.772744    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41022be5-27cd-437d-bd67-d2dd50bd61d1-tls-certs\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.772803 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.772781    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74wkd\" (UniqueName: \"kubernetes.io/projected/41022be5-27cd-437d-bd67-d2dd50bd61d1-kube-api-access-74wkd\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.772979 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.772807    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-tmp-dir\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.873859 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.873832    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-tmp-dir\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.874003 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.873893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-dshm\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.874003 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.873913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-model-cache\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.874003 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.873948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-home\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.874003 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.873962    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41022be5-27cd-437d-bd67-d2dd50bd61d1-tls-certs\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.874003 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.873977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74wkd\" (UniqueName: \"kubernetes.io/projected/41022be5-27cd-437d-bd67-d2dd50bd61d1-kube-api-access-74wkd\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.874282 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.874258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-tmp-dir\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.874353 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.874329    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-home\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.874429 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.874413    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-model-cache\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.876176 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.876158    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-dshm\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.876319 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.876300    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41022be5-27cd-437d-bd67-d2dd50bd61d1-tls-certs\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:49.880844 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:49.880824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74wkd\" (UniqueName: \"kubernetes.io/projected/41022be5-27cd-437d-bd67-d2dd50bd61d1-kube-api-access-74wkd\") pod \"e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") " pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:50.057991 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:50.057906    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:18:50.201093 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:50.201061    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"]
Jul 22 19:18:50.202663 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:18:50.202636    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41022be5_27cd_437d_bd67_d2dd50bd61d1.slice/crio-65fb2672afc084a3bff4d994d7f5172eaa431daef81dae20c903b8e680eabef1 WatchSource:0}: Error finding container 65fb2672afc084a3bff4d994d7f5172eaa431daef81dae20c903b8e680eabef1: Status 404 returned error can't find the container with id 65fb2672afc084a3bff4d994d7f5172eaa431daef81dae20c903b8e680eabef1
Jul 22 19:18:50.729042 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:50.729012    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2" event={"ID":"41022be5-27cd-437d-bd67-d2dd50bd61d1","Type":"ContainerStarted","Data":"9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886"}
Jul 22 19:18:50.729199 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:50.729053    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2" event={"ID":"41022be5-27cd-437d-bd67-d2dd50bd61d1","Type":"ContainerStarted","Data":"65fb2672afc084a3bff4d994d7f5172eaa431daef81dae20c903b8e680eabef1"}
Jul 22 19:18:50.746735 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:50.746691    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2" podStartSLOduration=1.746673048 podStartE2EDuration="1.746673048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:18:50.744732032 +0000 UTC m=+1680.776749561" watchObservedRunningTime="2026-07-22 19:18:50.746673048 +0000 UTC m=+1680.778690583"
Jul 22 19:18:53.257609 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:53.257523    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:53.257609 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:53.257566    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:53.269944 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:53.269924    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:18:53.755112 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:18:53.755079    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:19:00.058995 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:00.058962    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:19:00.059364 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:00.059008    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:19:00.071389 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:00.071366    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:19:00.141934 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:00.141902    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:19:00.795101 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:00.795073    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:19:16.261432 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:16.261343    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"]
Jul 22 19:19:17.477673 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:17.477644    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"]
Jul 22 19:19:19.934932 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:19.934902    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:19:31.281933 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.281872    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx" podUID="a7214303-86d1-4caf-8dac-51a0e38b0f8c" containerName="main" containerID="cri-o://53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9" gracePeriod=45
Jul 22 19:19:31.538601 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.538548    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:19:31.719816 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.719790    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-home\") pod \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") "
Jul 22 19:19:31.719962 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.719849    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tmp-dir\") pod \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") "
Jul 22 19:19:31.719962 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.719884    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-model-cache\") pod \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") "
Jul 22 19:19:31.719962 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.719924    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj8bg\" (UniqueName: \"kubernetes.io/projected/a7214303-86d1-4caf-8dac-51a0e38b0f8c-kube-api-access-tj8bg\") pod \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") "
Jul 22 19:19:31.720134 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.719963    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tls-certs\") pod \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") "
Jul 22 19:19:31.720134 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.719989    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-dshm\") pod \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\" (UID: \"a7214303-86d1-4caf-8dac-51a0e38b0f8c\") "
Jul 22 19:19:31.720134 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.720003    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-home" (OuterVolumeSpecName: "home") pod "a7214303-86d1-4caf-8dac-51a0e38b0f8c" (UID: "a7214303-86d1-4caf-8dac-51a0e38b0f8c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:19:31.720294 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.720138    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a7214303-86d1-4caf-8dac-51a0e38b0f8c" (UID: "a7214303-86d1-4caf-8dac-51a0e38b0f8c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:19:31.720294 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.720180    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-model-cache" (OuterVolumeSpecName: "model-cache") pod "a7214303-86d1-4caf-8dac-51a0e38b0f8c" (UID: "a7214303-86d1-4caf-8dac-51a0e38b0f8c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:19:31.720394 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.720338    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:31.720394 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.720358    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:31.720394 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.720372    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:31.722149 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.722118    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-dshm" (OuterVolumeSpecName: "dshm") pod "a7214303-86d1-4caf-8dac-51a0e38b0f8c" (UID: "a7214303-86d1-4caf-8dac-51a0e38b0f8c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:19:31.722252 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.722156    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a7214303-86d1-4caf-8dac-51a0e38b0f8c" (UID: "a7214303-86d1-4caf-8dac-51a0e38b0f8c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:19:31.722252 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.722186    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7214303-86d1-4caf-8dac-51a0e38b0f8c-kube-api-access-tj8bg" (OuterVolumeSpecName: "kube-api-access-tj8bg") pod "a7214303-86d1-4caf-8dac-51a0e38b0f8c" (UID: "a7214303-86d1-4caf-8dac-51a0e38b0f8c"). InnerVolumeSpecName "kube-api-access-tj8bg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:19:31.820975 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.820919    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj8bg\" (UniqueName: \"kubernetes.io/projected/a7214303-86d1-4caf-8dac-51a0e38b0f8c-kube-api-access-tj8bg\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:31.820975 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.820940    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7214303-86d1-4caf-8dac-51a0e38b0f8c-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:31.820975 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.820953    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7214303-86d1-4caf-8dac-51a0e38b0f8c-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:31.943659 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.943624    2558 generic.go:356] "Generic (PLEG): container finished" podID="a7214303-86d1-4caf-8dac-51a0e38b0f8c" containerID="53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9" exitCode=0
Jul 22 19:19:31.943789 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.943671    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"
Jul 22 19:19:31.943789 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.943710    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx" event={"ID":"a7214303-86d1-4caf-8dac-51a0e38b0f8c","Type":"ContainerDied","Data":"53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9"}
Jul 22 19:19:31.943789 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.943775    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx" event={"ID":"a7214303-86d1-4caf-8dac-51a0e38b0f8c","Type":"ContainerDied","Data":"e1bba102359b0354b960272dfad2ad2f23ae89a55a99ba0636454a6578c64019"}
Jul 22 19:19:31.943899 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.943797    2558 scope.go:117] "RemoveContainer" containerID="53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9"
Jul 22 19:19:31.952629 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.952614    2558 scope.go:117] "RemoveContainer" containerID="53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9"
Jul 22 19:19:31.952886 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:19:31.952868    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9\": container with ID starting with 53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9 not found: ID does not exist" containerID="53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9"
Jul 22 19:19:31.952946 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.952893    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9"} err="failed to get container status \"53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9\": rpc error: code = NotFound desc = could not find container \"53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9\": container with ID starting with 53de73045a05202dd7bbfdeb6f1db1c4d2c8febc72a58ba11262e5359286ffa9 not found: ID does not exist"
Jul 22 19:19:31.964285 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.964259    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"]
Jul 22 19:19:31.967822 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:31.967803    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-a69122a1-kserve-588f9bb647-r7qtx"]
Jul 22 19:19:32.137864 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.137839    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:19:32.451858 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.451786    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7214303-86d1-4caf-8dac-51a0e38b0f8c" path="/var/lib/kubelet/pods/a7214303-86d1-4caf-8dac-51a0e38b0f8c/volumes"
Jul 22 19:19:32.499725 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.499690    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2" podUID="41022be5-27cd-437d-bd67-d2dd50bd61d1" containerName="main" containerID="cri-o://9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886" gracePeriod=45
Jul 22 19:19:32.751095 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.751076    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:19:32.930496 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.930462    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-dshm\") pod \"41022be5-27cd-437d-bd67-d2dd50bd61d1\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") "
Jul 22 19:19:32.930496 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.930494    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-tmp-dir\") pod \"41022be5-27cd-437d-bd67-d2dd50bd61d1\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") "
Jul 22 19:19:32.930742 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.930597    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74wkd\" (UniqueName: \"kubernetes.io/projected/41022be5-27cd-437d-bd67-d2dd50bd61d1-kube-api-access-74wkd\") pod \"41022be5-27cd-437d-bd67-d2dd50bd61d1\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") "
Jul 22 19:19:32.930742 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.930640    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-home\") pod \"41022be5-27cd-437d-bd67-d2dd50bd61d1\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") "
Jul 22 19:19:32.930742 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.930659    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-model-cache\") pod \"41022be5-27cd-437d-bd67-d2dd50bd61d1\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") "
Jul 22 19:19:32.930742 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.930676    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41022be5-27cd-437d-bd67-d2dd50bd61d1-tls-certs\") pod \"41022be5-27cd-437d-bd67-d2dd50bd61d1\" (UID: \"41022be5-27cd-437d-bd67-d2dd50bd61d1\") "
Jul 22 19:19:32.930987 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.930854    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41022be5-27cd-437d-bd67-d2dd50bd61d1" (UID: "41022be5-27cd-437d-bd67-d2dd50bd61d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:19:32.930987 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.930936    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-home" (OuterVolumeSpecName: "home") pod "41022be5-27cd-437d-bd67-d2dd50bd61d1" (UID: "41022be5-27cd-437d-bd67-d2dd50bd61d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:19:32.930987 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.930957    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "41022be5-27cd-437d-bd67-d2dd50bd61d1" (UID: "41022be5-27cd-437d-bd67-d2dd50bd61d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:19:32.931147 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.931021    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:32.932585 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.932558    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-dshm" (OuterVolumeSpecName: "dshm") pod "41022be5-27cd-437d-bd67-d2dd50bd61d1" (UID: "41022be5-27cd-437d-bd67-d2dd50bd61d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:19:32.932717 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.932699    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41022be5-27cd-437d-bd67-d2dd50bd61d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41022be5-27cd-437d-bd67-d2dd50bd61d1" (UID: "41022be5-27cd-437d-bd67-d2dd50bd61d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:19:32.932787 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.932719    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41022be5-27cd-437d-bd67-d2dd50bd61d1-kube-api-access-74wkd" (OuterVolumeSpecName: "kube-api-access-74wkd") pod "41022be5-27cd-437d-bd67-d2dd50bd61d1" (UID: "41022be5-27cd-437d-bd67-d2dd50bd61d1"). InnerVolumeSpecName "kube-api-access-74wkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:19:32.950029 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.950005    2558 generic.go:356] "Generic (PLEG): container finished" podID="41022be5-27cd-437d-bd67-d2dd50bd61d1" containerID="9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886" exitCode=0
Jul 22 19:19:32.950133 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.950048    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"
Jul 22 19:19:32.950133 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.950056    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2" event={"ID":"41022be5-27cd-437d-bd67-d2dd50bd61d1","Type":"ContainerDied","Data":"9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886"}
Jul 22 19:19:32.950133 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.950095    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2" event={"ID":"41022be5-27cd-437d-bd67-d2dd50bd61d1","Type":"ContainerDied","Data":"65fb2672afc084a3bff4d994d7f5172eaa431daef81dae20c903b8e680eabef1"}
Jul 22 19:19:32.950133 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.950117    2558 scope.go:117] "RemoveContainer" containerID="9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886"
Jul 22 19:19:32.958390 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.958371    2558 scope.go:117] "RemoveContainer" containerID="9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886"
Jul 22 19:19:32.958637 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:19:32.958620    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886\": container with ID starting with 9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886 not found: ID does not exist" containerID="9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886"
Jul 22 19:19:32.958685 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.958645    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886"} err="failed to get container status \"9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886\": rpc error: code = NotFound desc = could not find container \"9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886\": container with ID starting with 9ee1e084dcbd118d9549cfbd9902e8fa5f066a95914ba555d7700b33f6f4d886 not found: ID does not exist"
Jul 22 19:19:32.969972 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.969942    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"]
Jul 22 19:19:32.971708 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:32.971690    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-a69122a1-kserve-57b94c85f9-ntrs2"]
Jul 22 19:19:33.031839 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:33.031767    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74wkd\" (UniqueName: \"kubernetes.io/projected/41022be5-27cd-437d-bd67-d2dd50bd61d1-kube-api-access-74wkd\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:33.031839 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:33.031796    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:33.031839 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:33.031810    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:33.031839 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:33.031821    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41022be5-27cd-437d-bd67-d2dd50bd61d1-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:33.031839 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:33.031831    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41022be5-27cd-437d-bd67-d2dd50bd61d1-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:19:34.452070 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:34.452034    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41022be5-27cd-437d-bd67-d2dd50bd61d1" path="/var/lib/kubelet/pods/41022be5-27cd-437d-bd67-d2dd50bd61d1/volumes"
Jul 22 19:19:39.543152 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:39.543117    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:19:55.136090 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:19:55.136057    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:20:05.841916 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:20:05.841882    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:20:14.740695 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:20:14.740661    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:20:23.634953 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:20:23.634915    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:20:33.847893 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:20:33.847856    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:20:43.335814 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:20:43.335718    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:20:57.743632 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:20:57.743598    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:21:06.839896 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:21:06.839857    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:21:16.038118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:21:16.038081    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:21:25.437559 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:21:25.437523    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:21:38.639781 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:21:38.639730    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:21:47.639286 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:21:47.639250    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:21:58.938022 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:21:58.937986    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:22:10.040120 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:22:10.040087    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:22:20.840600 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:22:20.840527    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:22:34.738204 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:22:34.738168    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:22:44.236203 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:22:44.236166    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:22:53.541984 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:22:53.541950    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:23:04.536491 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:04.536464    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:23:22.481967 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.481936    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"]
Jul 22 19:23:22.482400 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.482284    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41022be5-27cd-437d-bd67-d2dd50bd61d1" containerName="main"
Jul 22 19:23:22.482400 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.482294    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="41022be5-27cd-437d-bd67-d2dd50bd61d1" containerName="main"
Jul 22 19:23:22.482400 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.482308    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7214303-86d1-4caf-8dac-51a0e38b0f8c" containerName="main"
Jul 22 19:23:22.482400 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.482313    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7214303-86d1-4caf-8dac-51a0e38b0f8c" containerName="main"
Jul 22 19:23:22.482400 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.482380    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7214303-86d1-4caf-8dac-51a0e38b0f8c" containerName="main"
Jul 22 19:23:22.482400 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.482388    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="41022be5-27cd-437d-bd67-d2dd50bd61d1" containerName="main"
Jul 22 19:23:22.485437 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.485421    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"
Jul 22 19:23:22.487816 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.487794    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 19:23:22.487955 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.487828    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\""
Jul 22 19:23:22.491783 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.491761    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"]
Jul 22 19:23:22.574942 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.574920    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e815d974-34b5-409d-bcae-c45248516626-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8\" (UID: \"e815d974-34b5-409d-bcae-c45248516626\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"
Jul 22 19:23:22.675275 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.675249    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e815d974-34b5-409d-bcae-c45248516626-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8\" (UID: \"e815d974-34b5-409d-bcae-c45248516626\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"
Jul 22 19:23:22.675824 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.675809    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e815d974-34b5-409d-bcae-c45248516626-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8\" (UID: \"e815d974-34b5-409d-bcae-c45248516626\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"
Jul 22 19:23:22.796819 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.796764    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"
Jul 22 19:23:22.923658 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:22.923632    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"]
Jul 22 19:23:22.925726 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:23:22.925697    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode815d974_34b5_409d_bcae_c45248516626.slice/crio-de5571cb631a7e4d11469efbe2e8af0dfdfc228f28a9958320b084487e20e94c WatchSource:0}: Error finding container de5571cb631a7e4d11469efbe2e8af0dfdfc228f28a9958320b084487e20e94c: Status 404 returned error can't find the container with id de5571cb631a7e4d11469efbe2e8af0dfdfc228f28a9958320b084487e20e94c
Jul 22 19:23:23.085039 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:23.084979    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8" event={"ID":"e815d974-34b5-409d-bcae-c45248516626","Type":"ContainerStarted","Data":"de5571cb631a7e4d11469efbe2e8af0dfdfc228f28a9958320b084487e20e94c"}
Jul 22 19:23:24.091082 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:24.091048    2558 generic.go:356] "Generic (PLEG): container finished" podID="e815d974-34b5-409d-bcae-c45248516626" containerID="879a5abe7f1f9b799b18ee2b30d9f6ab82030b0c15805f334d609d5da59b7daf" exitCode=0
Jul 22 19:23:24.091524 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:24.091107    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8" event={"ID":"e815d974-34b5-409d-bcae-c45248516626","Type":"ContainerDied","Data":"879a5abe7f1f9b799b18ee2b30d9f6ab82030b0c15805f334d609d5da59b7daf"}
Jul 22 19:23:25.223122 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:25.223101    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"
Jul 22 19:23:25.296553 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:25.296528    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e815d974-34b5-409d-bcae-c45248516626-maas-api-service-ca\") pod \"e815d974-34b5-409d-bcae-c45248516626\" (UID: \"e815d974-34b5-409d-bcae-c45248516626\") "
Jul 22 19:23:25.296868 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:25.296847    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e815d974-34b5-409d-bcae-c45248516626-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e815d974-34b5-409d-bcae-c45248516626" (UID: "e815d974-34b5-409d-bcae-c45248516626"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:23:25.397415 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:25.397384    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e815d974-34b5-409d-bcae-c45248516626-maas-api-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:23:26.102675 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:26.102646    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"
Jul 22 19:23:26.102887 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:26.102646    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8" event={"ID":"e815d974-34b5-409d-bcae-c45248516626","Type":"ContainerDied","Data":"de5571cb631a7e4d11469efbe2e8af0dfdfc228f28a9958320b084487e20e94c"}
Jul 22 19:23:26.102887 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:26.102806    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de5571cb631a7e4d11469efbe2e8af0dfdfc228f28a9958320b084487e20e94c"
Jul 22 19:23:35.742901 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:35.742867    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:23:37.550626 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:37.550593    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"]
Jul 22 19:23:37.553242 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:37.553216    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b5a0ccd-rg5b8"]
Jul 22 19:23:38.451454 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:38.451418    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e815d974-34b5-409d-bcae-c45248516626" path="/var/lib/kubelet/pods/e815d974-34b5-409d-bcae-c45248516626/volumes"
Jul 22 19:23:47.243822 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:23:47.240521    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:24:06.243397 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.243369    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"]
Jul 22 19:24:06.243842 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.243763    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e815d974-34b5-409d-bcae-c45248516626" containerName="revoke-keys"
Jul 22 19:24:06.243842 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.243774    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e815d974-34b5-409d-bcae-c45248516626" containerName="revoke-keys"
Jul 22 19:24:06.243920 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.243846    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e815d974-34b5-409d-bcae-c45248516626" containerName="revoke-keys"
Jul 22 19:24:06.246781 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.246765    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"
Jul 22 19:24:06.249407 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.249382    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\""
Jul 22 19:24:06.249551 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.249428    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 19:24:06.254556 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.254531    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"]
Jul 22 19:24:06.320625 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.320596    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6f7f04-01e6-4332-9456-cf1e22306834-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e886b356-gwkll\" (UID: \"ff6f7f04-01e6-4332-9456-cf1e22306834\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"
Jul 22 19:24:06.421365 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.421341    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6f7f04-01e6-4332-9456-cf1e22306834-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e886b356-gwkll\" (UID: \"ff6f7f04-01e6-4332-9456-cf1e22306834\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"
Jul 22 19:24:06.421952 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.421935    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6f7f04-01e6-4332-9456-cf1e22306834-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e886b356-gwkll\" (UID: \"ff6f7f04-01e6-4332-9456-cf1e22306834\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"
Jul 22 19:24:06.557897 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.557821    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"
Jul 22 19:24:06.687960 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:06.687920    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"]
Jul 22 19:24:06.695623 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:24:06.695595    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6f7f04_01e6_4332_9456_cf1e22306834.slice/crio-a2d6e2f3a986a8b309cc7e2211ad25ed6f81e870afad28ea5d00b4f0702c1cc9 WatchSource:0}: Error finding container a2d6e2f3a986a8b309cc7e2211ad25ed6f81e870afad28ea5d00b4f0702c1cc9: Status 404 returned error can't find the container with id a2d6e2f3a986a8b309cc7e2211ad25ed6f81e870afad28ea5d00b4f0702c1cc9
Jul 22 19:24:07.310019 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:07.309986    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll" event={"ID":"ff6f7f04-01e6-4332-9456-cf1e22306834","Type":"ContainerStarted","Data":"1e24c1e3885631305c68163b4fcb789d30f8c8f6ee9af9a209074e77506fb7c3"}
Jul 22 19:24:07.310382 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:07.310041    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll" event={"ID":"ff6f7f04-01e6-4332-9456-cf1e22306834","Type":"ContainerStarted","Data":"a2d6e2f3a986a8b309cc7e2211ad25ed6f81e870afad28ea5d00b4f0702c1cc9"}
Jul 22 19:24:07.325273 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:07.325226    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll" podStartSLOduration=1.325210372 podStartE2EDuration="1.325210372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:24:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:24:07.32414391 +0000 UTC m=+1997.356161433" watchObservedRunningTime="2026-07-22 19:24:07.325210372 +0000 UTC m=+1997.357227901"
Jul 22 19:24:08.316441 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:08.316404    2558 generic.go:356] "Generic (PLEG): container finished" podID="ff6f7f04-01e6-4332-9456-cf1e22306834" containerID="1e24c1e3885631305c68163b4fcb789d30f8c8f6ee9af9a209074e77506fb7c3" exitCode=0
Jul 22 19:24:08.316848 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:08.316461    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll" event={"ID":"ff6f7f04-01e6-4332-9456-cf1e22306834","Type":"ContainerDied","Data":"1e24c1e3885631305c68163b4fcb789d30f8c8f6ee9af9a209074e77506fb7c3"}
Jul 22 19:24:09.453822 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:09.453798    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"
Jul 22 19:24:09.546440 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:09.546416    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6f7f04-01e6-4332-9456-cf1e22306834-maas-api-service-ca\") pod \"ff6f7f04-01e6-4332-9456-cf1e22306834\" (UID: \"ff6f7f04-01e6-4332-9456-cf1e22306834\") "
Jul 22 19:24:09.546799 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:09.546774    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff6f7f04-01e6-4332-9456-cf1e22306834-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff6f7f04-01e6-4332-9456-cf1e22306834" (UID: "ff6f7f04-01e6-4332-9456-cf1e22306834"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:24:09.647263 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:09.647234    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6f7f04-01e6-4332-9456-cf1e22306834-maas-api-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:24:10.327821 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:10.327788    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"
Jul 22 19:24:10.327821 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:10.327806    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll" event={"ID":"ff6f7f04-01e6-4332-9456-cf1e22306834","Type":"ContainerDied","Data":"a2d6e2f3a986a8b309cc7e2211ad25ed6f81e870afad28ea5d00b4f0702c1cc9"}
Jul 22 19:24:10.328037 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:10.327863    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2d6e2f3a986a8b309cc7e2211ad25ed6f81e870afad28ea5d00b4f0702c1cc9"
Jul 22 19:24:19.438603 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:19.438566    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:24:21.310300 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:21.310257    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"]
Jul 22 19:24:21.312802 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:21.312778    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e886b356-gwkll"]
Jul 22 19:24:22.451160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:22.451130    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff6f7f04-01e6-4332-9456-cf1e22306834" path="/var/lib/kubelet/pods/ff6f7f04-01e6-4332-9456-cf1e22306834/volumes"
Jul 22 19:24:24.487444 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.487415    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"]
Jul 22 19:24:24.487952 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.487812    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff6f7f04-01e6-4332-9456-cf1e22306834" containerName="revoke-keys"
Jul 22 19:24:24.487952 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.487825    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff6f7f04-01e6-4332-9456-cf1e22306834" containerName="revoke-keys"
Jul 22 19:24:24.487952 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.487889    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff6f7f04-01e6-4332-9456-cf1e22306834" containerName="revoke-keys"
Jul 22 19:24:24.491115 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.491096    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.493734 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.493707    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 19:24:24.493734 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.493727    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 19:24:24.493951 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.493710    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-3df5cf99-serving-cert\""
Jul 22 19:24:24.494695 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.494678    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65h52\""
Jul 22 19:24:24.501828 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.501806    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"]
Jul 22 19:24:24.564430 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.564403    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzwqm\" (UniqueName: \"kubernetes.io/projected/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-kube-api-access-zzwqm\") pod \"maas-api-e2e-ait-3df5cf99-54685c7fd-75j28\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.564544 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.564432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-maas-api-tls\") pod \"maas-api-e2e-ait-3df5cf99-54685c7fd-75j28\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.564544 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.564457    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-openshift-service-ca\") pod \"maas-api-e2e-ait-3df5cf99-54685c7fd-75j28\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.665729 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.665701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzwqm\" (UniqueName: \"kubernetes.io/projected/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-kube-api-access-zzwqm\") pod \"maas-api-e2e-ait-3df5cf99-54685c7fd-75j28\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.665862 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.665736    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-maas-api-tls\") pod \"maas-api-e2e-ait-3df5cf99-54685c7fd-75j28\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.665862 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.665783    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-openshift-service-ca\") pod \"maas-api-e2e-ait-3df5cf99-54685c7fd-75j28\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.666484 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.666467    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-openshift-service-ca\") pod \"maas-api-e2e-ait-3df5cf99-54685c7fd-75j28\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.668218 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.668199    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-maas-api-tls\") pod \"maas-api-e2e-ait-3df5cf99-54685c7fd-75j28\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.673894 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.673866    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzwqm\" (UniqueName: \"kubernetes.io/projected/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-kube-api-access-zzwqm\") pod \"maas-api-e2e-ait-3df5cf99-54685c7fd-75j28\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.803774 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.803673    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:24.942145 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.942117    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"]
Jul 22 19:24:24.942588 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:24:24.942570    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb397b7a1_ef97_411b_a1ca_9b5dbbc74b3c.slice/crio-f33de791e7adb9196b86e7545a525eed9377350ec2c8cba8bbad696e3bbbc640 WatchSource:0}: Error finding container f33de791e7adb9196b86e7545a525eed9377350ec2c8cba8bbad696e3bbbc640: Status 404 returned error can't find the container with id f33de791e7adb9196b86e7545a525eed9377350ec2c8cba8bbad696e3bbbc640
Jul 22 19:24:24.943972 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:24.943953    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 19:24:25.405904 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:25.405866    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28" event={"ID":"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c","Type":"ContainerStarted","Data":"f33de791e7adb9196b86e7545a525eed9377350ec2c8cba8bbad696e3bbbc640"}
Jul 22 19:24:27.417904 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:27.417866    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28" event={"ID":"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c","Type":"ContainerStarted","Data":"445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2"}
Jul 22 19:24:27.418270 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:27.417955    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:27.434415 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:27.434374    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28" podStartSLOduration=1.150238852 podStartE2EDuration="3.434359699s" totalImagesPullingTime="2.284120847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:24:27.433014359 +0000 UTC m=+2017.465031890" watchObservedRunningTime="2026-07-22 19:24:27.434359699 +0000 UTC m=+2017.466377207"
Jul 22 19:24:32.335053 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:32.335019    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:24:33.434508 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:33.434478    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:24:50.195089 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.195054    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"]
Jul 22 19:24:50.198824 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.198805    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"
Jul 22 19:24:50.201111 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.201090    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\""
Jul 22 19:24:50.204829 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.204810    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"]
Jul 22 19:24:50.375514 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.375469    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b261ebb6-f6ba-4ca4-b961-470bdf51b4a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz\" (UID: \"b261ebb6-f6ba-4ca4-b961-470bdf51b4a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"
Jul 22 19:24:50.476153 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.476066    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b261ebb6-f6ba-4ca4-b961-470bdf51b4a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz\" (UID: \"b261ebb6-f6ba-4ca4-b961-470bdf51b4a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"
Jul 22 19:24:50.476708 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.476689    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b261ebb6-f6ba-4ca4-b961-470bdf51b4a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz\" (UID: \"b261ebb6-f6ba-4ca4-b961-470bdf51b4a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"
Jul 22 19:24:50.513032 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.513004    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\""
Jul 22 19:24:50.521836 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.521817    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"
Jul 22 19:24:50.860172 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:50.859948    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"]
Jul 22 19:24:50.865310 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:24:50.862728    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb261ebb6_f6ba_4ca4_b961_470bdf51b4a5.slice/crio-d787aaf25aa8906e6c881b95d4e961bddd58e8d6f8ca80d105a9f309731b0068 WatchSource:0}: Error finding container d787aaf25aa8906e6c881b95d4e961bddd58e8d6f8ca80d105a9f309731b0068: Status 404 returned error can't find the container with id d787aaf25aa8906e6c881b95d4e961bddd58e8d6f8ca80d105a9f309731b0068
Jul 22 19:24:51.543244 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:51.543154    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz" event={"ID":"b261ebb6-f6ba-4ca4-b961-470bdf51b4a5","Type":"ContainerStarted","Data":"839207347b36ab52fd8e0e681c4acef90a9a5270e944a594a87ac58f6883f912"}
Jul 22 19:24:51.543244 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:51.543215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz" event={"ID":"b261ebb6-f6ba-4ca4-b961-470bdf51b4a5","Type":"ContainerStarted","Data":"d787aaf25aa8906e6c881b95d4e961bddd58e8d6f8ca80d105a9f309731b0068"}
Jul 22 19:24:51.557085 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:51.557039    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz" podStartSLOduration=1.557024466 podStartE2EDuration="1.557024466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:24:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:24:51.556315306 +0000 UTC m=+2041.588332836" watchObservedRunningTime="2026-07-22 19:24:51.557024466 +0000 UTC m=+2041.589041995"
Jul 22 19:24:52.549718 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:52.549678    2558 generic.go:356] "Generic (PLEG): container finished" podID="b261ebb6-f6ba-4ca4-b961-470bdf51b4a5" containerID="839207347b36ab52fd8e0e681c4acef90a9a5270e944a594a87ac58f6883f912" exitCode=0
Jul 22 19:24:52.550169 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:52.549778    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz" event={"ID":"b261ebb6-f6ba-4ca4-b961-470bdf51b4a5","Type":"ContainerDied","Data":"839207347b36ab52fd8e0e681c4acef90a9a5270e944a594a87ac58f6883f912"}
Jul 22 19:24:53.692840 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:53.692812    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"
Jul 22 19:24:53.704737 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:53.704717    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b261ebb6-f6ba-4ca4-b961-470bdf51b4a5-maas-api-service-ca\") pod \"b261ebb6-f6ba-4ca4-b961-470bdf51b4a5\" (UID: \"b261ebb6-f6ba-4ca4-b961-470bdf51b4a5\") "
Jul 22 19:24:53.705040 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:53.705021    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b261ebb6-f6ba-4ca4-b961-470bdf51b4a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b261ebb6-f6ba-4ca4-b961-470bdf51b4a5" (UID: "b261ebb6-f6ba-4ca4-b961-470bdf51b4a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:24:53.806118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:53.806057    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b261ebb6-f6ba-4ca4-b961-470bdf51b4a5-maas-api-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:24:54.561064 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:54.561038    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"
Jul 22 19:24:54.561216 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:54.561043    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz" event={"ID":"b261ebb6-f6ba-4ca4-b961-470bdf51b4a5","Type":"ContainerDied","Data":"d787aaf25aa8906e6c881b95d4e961bddd58e8d6f8ca80d105a9f309731b0068"}
Jul 22 19:24:54.561216 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:24:54.561156    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d787aaf25aa8906e6c881b95d4e961bddd58e8d6f8ca80d105a9f309731b0068"
Jul 22 19:25:00.564507 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.564474    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"]
Jul 22 19:25:00.565086 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.564731    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28" podUID="b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c" containerName="maas-api" containerID="cri-o://445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2" gracePeriod=30
Jul 22 19:25:00.814715 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.814648    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:25:00.865125 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.865098    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-openshift-service-ca\") pod \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") "
Jul 22 19:25:00.865273 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.865204    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-maas-api-tls\") pod \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") "
Jul 22 19:25:00.865273 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.865225    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzwqm\" (UniqueName: \"kubernetes.io/projected/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-kube-api-access-zzwqm\") pod \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\" (UID: \"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c\") "
Jul 22 19:25:00.865495 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.865466    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c" (UID: "b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:25:00.867310 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.867286    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c" (UID: "b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:25:00.867420 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.867342    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-kube-api-access-zzwqm" (OuterVolumeSpecName: "kube-api-access-zzwqm") pod "b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c" (UID: "b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c"). InnerVolumeSpecName "kube-api-access-zzwqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:25:00.966556 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.966516    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:25:00.966556 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.966554    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzwqm\" (UniqueName: \"kubernetes.io/projected/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-kube-api-access-zzwqm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:25:00.966556 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:00.966565    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:25:01.598733 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:01.598700    2558 generic.go:356] "Generic (PLEG): container finished" podID="b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c" containerID="445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2" exitCode=0
Jul 22 19:25:01.599159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:01.598743    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"
Jul 22 19:25:01.599159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:01.598781    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28" event={"ID":"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c","Type":"ContainerDied","Data":"445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2"}
Jul 22 19:25:01.599159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:01.598833    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28" event={"ID":"b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c","Type":"ContainerDied","Data":"f33de791e7adb9196b86e7545a525eed9377350ec2c8cba8bbad696e3bbbc640"}
Jul 22 19:25:01.599159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:01.598856    2558 scope.go:117] "RemoveContainer" containerID="445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2"
Jul 22 19:25:01.608332 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:01.608311    2558 scope.go:117] "RemoveContainer" containerID="445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2"
Jul 22 19:25:01.608580 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:25:01.608563    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2\": container with ID starting with 445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2 not found: ID does not exist" containerID="445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2"
Jul 22 19:25:01.608622 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:01.608589    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2"} err="failed to get container status \"445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2\": rpc error: code = NotFound desc = could not find container \"445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2\": container with ID starting with 445d3feb11a4b0b9230b45748cc3fb1cdbea196752b479d1b9c7a6f335303cc2 not found: ID does not exist"
Jul 22 19:25:01.619319 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:01.619294    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"]
Jul 22 19:25:01.621478 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:01.621457    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3df5cf99-54685c7fd-75j28"]
Jul 22 19:25:02.456300 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:02.456264    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c" path="/var/lib/kubelet/pods/b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c/volumes"
Jul 22 19:25:03.244722 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:03.244688    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:25:05.263193 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:05.263161    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"]
Jul 22 19:25:05.265805 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:05.265780    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3df5cf99-g9mcz"]
Jul 22 19:25:06.451344 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:06.451308    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b261ebb6-f6ba-4ca4-b961-470bdf51b4a5" path="/var/lib/kubelet/pods/b261ebb6-f6ba-4ca4-b961-470bdf51b4a5/volumes"
Jul 22 19:25:09.771816 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.771784    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"]
Jul 22 19:25:09.772375 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.772351    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c" containerName="maas-api"
Jul 22 19:25:09.772521 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.772424    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c" containerName="maas-api"
Jul 22 19:25:09.772521 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.772458    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b261ebb6-f6ba-4ca4-b961-470bdf51b4a5" containerName="revoke-keys"
Jul 22 19:25:09.772521 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.772467    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b261ebb6-f6ba-4ca4-b961-470bdf51b4a5" containerName="revoke-keys"
Jul 22 19:25:09.772722 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.772576    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b397b7a1-ef97-411b-a1ca-9b5dbbc74b3c" containerName="maas-api"
Jul 22 19:25:09.772722 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.772593    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b261ebb6-f6ba-4ca4-b961-470bdf51b4a5" containerName="revoke-keys"
Jul 22 19:25:09.777035 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.777018    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:09.779845 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.779821    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 19:25:09.779972 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.779867    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 19:25:09.779972 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.779894    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65h52\""
Jul 22 19:25:09.779972 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.779947    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-bb81af95-serving-cert\""
Jul 22 19:25:09.786100 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.786042    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"]
Jul 22 19:25:09.837883 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.837856    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e686395a-eacd-47b7-bd79-96ed70568493-maas-api-tls\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:09.838010 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.837897    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e686395a-eacd-47b7-bd79-96ed70568493-openshift-service-ca\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:09.838010 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.837918    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pqvg\" (UniqueName: \"kubernetes.io/projected/e686395a-eacd-47b7-bd79-96ed70568493-kube-api-access-2pqvg\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:09.939304 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.939279    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e686395a-eacd-47b7-bd79-96ed70568493-maas-api-tls\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:09.939439 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.939317    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e686395a-eacd-47b7-bd79-96ed70568493-openshift-service-ca\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:09.939439 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.939336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pqvg\" (UniqueName: \"kubernetes.io/projected/e686395a-eacd-47b7-bd79-96ed70568493-kube-api-access-2pqvg\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:09.939562 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:25:09.939444    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-serving-cert: secret "maas-api-e2e-derive-bb81af95-serving-cert" not found
Jul 22 19:25:09.939562 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:25:09.939532    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e686395a-eacd-47b7-bd79-96ed70568493-maas-api-tls podName:e686395a-eacd-47b7-bd79-96ed70568493 nodeName:}" failed. No retries permitted until 2026-07-22 19:25:10.439509209 +0000 UTC m=+2060.471526730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e686395a-eacd-47b7-bd79-96ed70568493-maas-api-tls") pod "maas-api-e2e-derive-bb81af95-6467d97d94-sszd2" (UID: "e686395a-eacd-47b7-bd79-96ed70568493") : secret "maas-api-e2e-derive-bb81af95-serving-cert" not found
Jul 22 19:25:09.940224 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.940197    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e686395a-eacd-47b7-bd79-96ed70568493-openshift-service-ca\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:09.949741 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:09.949715    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pqvg\" (UniqueName: \"kubernetes.io/projected/e686395a-eacd-47b7-bd79-96ed70568493-kube-api-access-2pqvg\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:10.443242 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:10.443211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e686395a-eacd-47b7-bd79-96ed70568493-maas-api-tls\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:10.445630 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:10.445600    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e686395a-eacd-47b7-bd79-96ed70568493-maas-api-tls\") pod \"maas-api-e2e-derive-bb81af95-6467d97d94-sszd2\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:10.689220 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:10.689181    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:10.819091 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:10.819069    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"]
Jul 22 19:25:10.821202 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:25:10.821176    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode686395a_eacd_47b7_bd79_96ed70568493.slice/crio-3dc5657a7733754025e418aa8f3b1bc7bfb5c70da52a2ca8b3a99858b974d733 WatchSource:0}: Error finding container 3dc5657a7733754025e418aa8f3b1bc7bfb5c70da52a2ca8b3a99858b974d733: Status 404 returned error can't find the container with id 3dc5657a7733754025e418aa8f3b1bc7bfb5c70da52a2ca8b3a99858b974d733
Jul 22 19:25:11.649787 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:11.649728    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2" event={"ID":"e686395a-eacd-47b7-bd79-96ed70568493","Type":"ContainerStarted","Data":"103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64"}
Jul 22 19:25:11.649976 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:11.649811    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2" event={"ID":"e686395a-eacd-47b7-bd79-96ed70568493","Type":"ContainerStarted","Data":"3dc5657a7733754025e418aa8f3b1bc7bfb5c70da52a2ca8b3a99858b974d733"}
Jul 22 19:25:11.649976 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:11.649868    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:11.667070 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:11.667028    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2" podStartSLOduration=2.276189644 podStartE2EDuration="2.667014826s" totalImagesPullingTime="390.825182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:25:11.66653629 +0000 UTC m=+2061.698553844" watchObservedRunningTime="2026-07-22 19:25:11.667014826 +0000 UTC m=+2061.699032355"
Jul 22 19:25:15.642917 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:15.642845    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:25:17.660572 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:17.660544    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:34.671724 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:34.671683    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"]
Jul 22 19:25:34.675263 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:34.675243    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"
Jul 22 19:25:34.677455 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:34.677436    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\""
Jul 22 19:25:34.681708 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:34.681684    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"]
Jul 22 19:25:34.836105 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:34.836073    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab08785a-f06e-46e8-a503-92b14c17c27b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5\" (UID: \"ab08785a-f06e-46e8-a503-92b14c17c27b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"
Jul 22 19:25:34.936653 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:34.936591    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab08785a-f06e-46e8-a503-92b14c17c27b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5\" (UID: \"ab08785a-f06e-46e8-a503-92b14c17c27b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"
Jul 22 19:25:34.937220 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:34.937202    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab08785a-f06e-46e8-a503-92b14c17c27b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5\" (UID: \"ab08785a-f06e-46e8-a503-92b14c17c27b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"
Jul 22 19:25:34.987622 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:34.987602    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"
Jul 22 19:25:35.315831 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:35.315807    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"]
Jul 22 19:25:35.317700 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:25:35.317673    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab08785a_f06e_46e8_a503_92b14c17c27b.slice/crio-1193beb5786cec226b253d876883c3d56b1ade31f0f1b64ca826b620d0c94472 WatchSource:0}: Error finding container 1193beb5786cec226b253d876883c3d56b1ade31f0f1b64ca826b620d0c94472: Status 404 returned error can't find the container with id 1193beb5786cec226b253d876883c3d56b1ade31f0f1b64ca826b620d0c94472
Jul 22 19:25:35.771523 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:35.771496    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5" event={"ID":"ab08785a-f06e-46e8-a503-92b14c17c27b","Type":"ContainerStarted","Data":"c11e9ffad959f85212f5995684bfa2b96c68099425f77fde12003be368dc3552"}
Jul 22 19:25:35.771962 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:35.771550    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5" event={"ID":"ab08785a-f06e-46e8-a503-92b14c17c27b","Type":"ContainerStarted","Data":"1193beb5786cec226b253d876883c3d56b1ade31f0f1b64ca826b620d0c94472"}
Jul 22 19:25:35.787194 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:35.787151    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5" podStartSLOduration=1.787138396 podStartE2EDuration="1.787138396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:25:35.786138952 +0000 UTC m=+2085.818156483" watchObservedRunningTime="2026-07-22 19:25:35.787138396 +0000 UTC m=+2085.819155924"
Jul 22 19:25:36.777943 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:36.777905    2558 generic.go:356] "Generic (PLEG): container finished" podID="ab08785a-f06e-46e8-a503-92b14c17c27b" containerID="c11e9ffad959f85212f5995684bfa2b96c68099425f77fde12003be368dc3552" exitCode=0
Jul 22 19:25:36.778376 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:36.777986    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5" event={"ID":"ab08785a-f06e-46e8-a503-92b14c17c27b","Type":"ContainerDied","Data":"c11e9ffad959f85212f5995684bfa2b96c68099425f77fde12003be368dc3552"}
Jul 22 19:25:37.919761 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:37.919724    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"
Jul 22 19:25:38.061370 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:38.061275    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab08785a-f06e-46e8-a503-92b14c17c27b-maas-api-service-ca\") pod \"ab08785a-f06e-46e8-a503-92b14c17c27b\" (UID: \"ab08785a-f06e-46e8-a503-92b14c17c27b\") "
Jul 22 19:25:38.061638 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:38.061616    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab08785a-f06e-46e8-a503-92b14c17c27b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab08785a-f06e-46e8-a503-92b14c17c27b" (UID: "ab08785a-f06e-46e8-a503-92b14c17c27b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:25:38.162203 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:38.162175    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab08785a-f06e-46e8-a503-92b14c17c27b-maas-api-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:25:38.789758 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:38.789651    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5" event={"ID":"ab08785a-f06e-46e8-a503-92b14c17c27b","Type":"ContainerDied","Data":"1193beb5786cec226b253d876883c3d56b1ade31f0f1b64ca826b620d0c94472"}
Jul 22 19:25:38.789758 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:38.789693    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"
Jul 22 19:25:38.789980 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:38.789703    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1193beb5786cec226b253d876883c3d56b1ade31f0f1b64ca826b620d0c94472"
Jul 22 19:25:44.868118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:44.868082    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"]
Jul 22 19:25:44.868567 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:44.868320    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2" podUID="e686395a-eacd-47b7-bd79-96ed70568493" containerName="maas-api" containerID="cri-o://103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64" gracePeriod=30
Jul 22 19:25:45.122493 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.122438    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:45.218213 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.218191    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e686395a-eacd-47b7-bd79-96ed70568493-openshift-service-ca\") pod \"e686395a-eacd-47b7-bd79-96ed70568493\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") "
Jul 22 19:25:45.218324 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.218240    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pqvg\" (UniqueName: \"kubernetes.io/projected/e686395a-eacd-47b7-bd79-96ed70568493-kube-api-access-2pqvg\") pod \"e686395a-eacd-47b7-bd79-96ed70568493\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") "
Jul 22 19:25:45.218324 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.218302    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e686395a-eacd-47b7-bd79-96ed70568493-maas-api-tls\") pod \"e686395a-eacd-47b7-bd79-96ed70568493\" (UID: \"e686395a-eacd-47b7-bd79-96ed70568493\") "
Jul 22 19:25:45.218506 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.218484    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e686395a-eacd-47b7-bd79-96ed70568493-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e686395a-eacd-47b7-bd79-96ed70568493" (UID: "e686395a-eacd-47b7-bd79-96ed70568493"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:25:45.220180 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.220141    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e686395a-eacd-47b7-bd79-96ed70568493-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e686395a-eacd-47b7-bd79-96ed70568493" (UID: "e686395a-eacd-47b7-bd79-96ed70568493"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:25:45.220313 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.220294    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e686395a-eacd-47b7-bd79-96ed70568493-kube-api-access-2pqvg" (OuterVolumeSpecName: "kube-api-access-2pqvg") pod "e686395a-eacd-47b7-bd79-96ed70568493" (UID: "e686395a-eacd-47b7-bd79-96ed70568493"). InnerVolumeSpecName "kube-api-access-2pqvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:25:45.319616 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.319589    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e686395a-eacd-47b7-bd79-96ed70568493-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:25:45.319616 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.319613    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pqvg\" (UniqueName: \"kubernetes.io/projected/e686395a-eacd-47b7-bd79-96ed70568493-kube-api-access-2pqvg\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:25:45.319790 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.319623    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e686395a-eacd-47b7-bd79-96ed70568493-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:25:45.827933 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.827897    2558 generic.go:356] "Generic (PLEG): container finished" podID="e686395a-eacd-47b7-bd79-96ed70568493" containerID="103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64" exitCode=0
Jul 22 19:25:45.828100 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.827940    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"
Jul 22 19:25:45.828100 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.827982    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2" event={"ID":"e686395a-eacd-47b7-bd79-96ed70568493","Type":"ContainerDied","Data":"103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64"}
Jul 22 19:25:45.828100 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.828034    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2" event={"ID":"e686395a-eacd-47b7-bd79-96ed70568493","Type":"ContainerDied","Data":"3dc5657a7733754025e418aa8f3b1bc7bfb5c70da52a2ca8b3a99858b974d733"}
Jul 22 19:25:45.828100 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.828057    2558 scope.go:117] "RemoveContainer" containerID="103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64"
Jul 22 19:25:45.837343 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.837326    2558 scope.go:117] "RemoveContainer" containerID="103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64"
Jul 22 19:25:45.837582 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:25:45.837563    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64\": container with ID starting with 103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64 not found: ID does not exist" containerID="103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64"
Jul 22 19:25:45.837648 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.837590    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64"} err="failed to get container status \"103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64\": rpc error: code = NotFound desc = could not find container \"103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64\": container with ID starting with 103cf029355055d686cea3c0bdfaab3e89abbb2fb42838b756ff4bfee15c7f64 not found: ID does not exist"
Jul 22 19:25:45.848743 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.848721    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"]
Jul 22 19:25:45.852463 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:45.852445    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-bb81af95-6467d97d94-sszd2"]
Jul 22 19:25:46.451667 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:46.451635    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e686395a-eacd-47b7-bd79-96ed70568493" path="/var/lib/kubelet/pods/e686395a-eacd-47b7-bd79-96ed70568493/volumes"
Jul 22 19:25:47.637988 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:47.637957    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:25:49.758608 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:49.758576    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"]
Jul 22 19:25:49.761159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:49.761135    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bb81af95-gvrx5"]
Jul 22 19:25:50.451875 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:25:50.451838    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab08785a-f06e-46e8-a503-92b14c17c27b" path="/var/lib/kubelet/pods/ab08785a-f06e-46e8-a503-92b14c17c27b/volumes"
Jul 22 19:27:20.945793 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:20.945742    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:27:31.146188 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:31.146152    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:27:35.098358 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.098321    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"]
Jul 22 19:27:35.098715 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.098701    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab08785a-f06e-46e8-a503-92b14c17c27b" containerName="revoke-keys"
Jul 22 19:27:35.098715 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.098716    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab08785a-f06e-46e8-a503-92b14c17c27b" containerName="revoke-keys"
Jul 22 19:27:35.098813 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.098734    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e686395a-eacd-47b7-bd79-96ed70568493" containerName="maas-api"
Jul 22 19:27:35.098813 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.098740    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e686395a-eacd-47b7-bd79-96ed70568493" containerName="maas-api"
Jul 22 19:27:35.098877 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.098816    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab08785a-f06e-46e8-a503-92b14c17c27b" containerName="revoke-keys"
Jul 22 19:27:35.098877 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.098826    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e686395a-eacd-47b7-bd79-96ed70568493" containerName="maas-api"
Jul 22 19:27:35.102084 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.102064    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.104823 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.104787    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-890e74a3-serving-cert\""
Jul 22 19:27:35.104966 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.104866    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65h52\""
Jul 22 19:27:35.104966 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.104882    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 19:27:35.105789 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.105772    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 19:27:35.112255 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.112232    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"]
Jul 22 19:27:35.220158 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.220130    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-openshift-service-ca\") pod \"maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.220296 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.220171    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-maas-api-tls\") pod \"maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.220296 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.220256    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x49fc\" (UniqueName: \"kubernetes.io/projected/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-kube-api-access-x49fc\") pod \"maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.321294 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.321267    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-openshift-service-ca\") pod \"maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.321437 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.321308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-maas-api-tls\") pod \"maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.321437 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.321356    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x49fc\" (UniqueName: \"kubernetes.io/projected/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-kube-api-access-x49fc\") pod \"maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.322135 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.322106    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-openshift-service-ca\") pod \"maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.323741 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.323719    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-maas-api-tls\") pod \"maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.329814 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.329787    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x49fc\" (UniqueName: \"kubernetes.io/projected/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-kube-api-access-x49fc\") pod \"maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.413564 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.413542    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:35.552963 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:35.552939    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"]
Jul 22 19:27:35.554897 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:27:35.554875    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d4c7a1_6a52_4c42_aa1d_6894c96cb5b9.slice/crio-3699667ac4aad92718fb89178bd7c9a018de12080fdf70e0ca95f3f565dfc4b3 WatchSource:0}: Error finding container 3699667ac4aad92718fb89178bd7c9a018de12080fdf70e0ca95f3f565dfc4b3: Status 404 returned error can't find the container with id 3699667ac4aad92718fb89178bd7c9a018de12080fdf70e0ca95f3f565dfc4b3
Jul 22 19:27:36.378155 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:36.378118    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl" event={"ID":"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9","Type":"ContainerStarted","Data":"9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc"}
Jul 22 19:27:36.378505 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:36.378207    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:36.378505 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:36.378223    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl" event={"ID":"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9","Type":"ContainerStarted","Data":"3699667ac4aad92718fb89178bd7c9a018de12080fdf70e0ca95f3f565dfc4b3"}
Jul 22 19:27:36.396064 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:36.396014    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl" podStartSLOduration=0.804763897 podStartE2EDuration="1.395994837s" totalImagesPullingTime="591.23094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:27:36.395715447 +0000 UTC m=+2206.427732978" watchObservedRunningTime="2026-07-22 19:27:36.395994837 +0000 UTC m=+2206.428012370"
Jul 22 19:27:42.390231 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:42.390198    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:43.041951 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:43.041916    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:27:46.492590 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:46.492552    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"]
Jul 22 19:27:46.496480 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:46.496463    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"
Jul 22 19:27:46.499016 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:46.498995    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\""
Jul 22 19:27:46.502736 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:46.502716    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"]
Jul 22 19:27:46.514781 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:46.514737    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf\" (UID: \"8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"
Jul 22 19:27:46.616117 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:46.616089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf\" (UID: \"8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"
Jul 22 19:27:46.616734 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:46.616711    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf\" (UID: \"8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"
Jul 22 19:27:46.809200 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:46.809131    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"
Jul 22 19:27:46.939772 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:46.939734    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"]
Jul 22 19:27:46.941997 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:27:46.941972    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd9de2d_32e7_49b5_b4b6_63f3f8b41c9c.slice/crio-6b864d2de470121d45e689ab67bf3880cdd8a7028134f7a83f0adc8bd2999922 WatchSource:0}: Error finding container 6b864d2de470121d45e689ab67bf3880cdd8a7028134f7a83f0adc8bd2999922: Status 404 returned error can't find the container with id 6b864d2de470121d45e689ab67bf3880cdd8a7028134f7a83f0adc8bd2999922
Jul 22 19:27:47.443032 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:47.442997    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf" event={"ID":"8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c","Type":"ContainerStarted","Data":"186f485dcb37089b1e63cbfa3c22d94ebf55cbc3f43cb3c39a1ab86ce837c144"}
Jul 22 19:27:47.443178 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:47.443049    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf" event={"ID":"8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c","Type":"ContainerStarted","Data":"6b864d2de470121d45e689ab67bf3880cdd8a7028134f7a83f0adc8bd2999922"}
Jul 22 19:27:47.458226 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:47.458182    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf" podStartSLOduration=1.45816685 podStartE2EDuration="1.45816685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:27:47.457595892 +0000 UTC m=+2217.489613421" watchObservedRunningTime="2026-07-22 19:27:47.45816685 +0000 UTC m=+2217.490184387"
Jul 22 19:27:48.449668 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:48.449636    2558 generic.go:356] "Generic (PLEG): container finished" podID="8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c" containerID="186f485dcb37089b1e63cbfa3c22d94ebf55cbc3f43cb3c39a1ab86ce837c144" exitCode=0
Jul 22 19:27:48.456776 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:48.454835    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf" event={"ID":"8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c","Type":"ContainerDied","Data":"186f485dcb37089b1e63cbfa3c22d94ebf55cbc3f43cb3c39a1ab86ce837c144"}
Jul 22 19:27:49.590486 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:49.590459    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"
Jul 22 19:27:49.636721 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:49.636697    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c-maas-api-service-ca\") pod \"8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c\" (UID: \"8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c\") "
Jul 22 19:27:49.637047 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:49.637022    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c" (UID: "8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:27:49.737882 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:49.737827    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c-maas-api-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:27:50.459968 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:50.459942    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"
Jul 22 19:27:50.460107 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:50.459965    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf" event={"ID":"8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c","Type":"ContainerDied","Data":"6b864d2de470121d45e689ab67bf3880cdd8a7028134f7a83f0adc8bd2999922"}
Jul 22 19:27:50.460107 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:50.460011    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b864d2de470121d45e689ab67bf3880cdd8a7028134f7a83f0adc8bd2999922"
Jul 22 19:27:56.882965 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:56.882930    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"]
Jul 22 19:27:56.883480 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:56.883180    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl" podUID="b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9" containerName="maas-api" containerID="cri-o://9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc" gracePeriod=30
Jul 22 19:27:57.132051 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.132023    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:57.200123 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.200066    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-maas-api-tls\") pod \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") "
Jul 22 19:27:57.200237 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.200139    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-openshift-service-ca\") pod \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") "
Jul 22 19:27:57.200237 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.200180    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x49fc\" (UniqueName: \"kubernetes.io/projected/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-kube-api-access-x49fc\") pod \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\" (UID: \"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9\") "
Jul 22 19:27:57.200500 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.200465    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9" (UID: "b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:27:57.202065 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.202043    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9" (UID: "b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:27:57.202201 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.202181    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-kube-api-access-x49fc" (OuterVolumeSpecName: "kube-api-access-x49fc") pod "b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9" (UID: "b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9"). InnerVolumeSpecName "kube-api-access-x49fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:27:57.301102 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.301083    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:27:57.301102 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.301103    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x49fc\" (UniqueName: \"kubernetes.io/projected/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-kube-api-access-x49fc\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:27:57.301230 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.301113    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:27:57.500911 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.500826    2558 generic.go:356] "Generic (PLEG): container finished" podID="b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9" containerID="9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc" exitCode=0
Jul 22 19:27:57.500911 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.500874    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"
Jul 22 19:27:57.501101 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.500916    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl" event={"ID":"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9","Type":"ContainerDied","Data":"9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc"}
Jul 22 19:27:57.501101 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.500964    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl" event={"ID":"b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9","Type":"ContainerDied","Data":"3699667ac4aad92718fb89178bd7c9a018de12080fdf70e0ca95f3f565dfc4b3"}
Jul 22 19:27:57.501101 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.500991    2558 scope.go:117] "RemoveContainer" containerID="9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc"
Jul 22 19:27:57.510290 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.510272    2558 scope.go:117] "RemoveContainer" containerID="9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc"
Jul 22 19:27:57.510522 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:27:57.510501    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc\": container with ID starting with 9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc not found: ID does not exist" containerID="9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc"
Jul 22 19:27:57.510568 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.510531    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc"} err="failed to get container status \"9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc\": rpc error: code = NotFound desc = could not find container \"9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc\": container with ID starting with 9eafd4bc1d38f81733a0daa1609c8332c888960aa0b407f5bfffd8ed0882b5cc not found: ID does not exist"
Jul 22 19:27:57.521731 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.521711    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"]
Jul 22 19:27:57.526837 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:57.526813    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-890e74a3-64b8dcd545-vr7sl"]
Jul 22 19:27:58.451832 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:58.451799    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9" path="/var/lib/kubelet/pods/b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9/volumes"
Jul 22 19:27:59.744780 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:27:59.744737    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:28:01.568131 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:01.568098    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"]
Jul 22 19:28:01.571210 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:01.571181    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-890e74a3-vr7sf"]
Jul 22 19:28:02.451985 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:02.451951    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c" path="/var/lib/kubelet/pods/8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c/volumes"
Jul 22 19:28:30.342896 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.342814    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"]
Jul 22 19:28:30.343344 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.343211    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9" containerName="maas-api"
Jul 22 19:28:30.343344 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.343225    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9" containerName="maas-api"
Jul 22 19:28:30.343344 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.343245    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c" containerName="revoke-keys"
Jul 22 19:28:30.343344 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.343251    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c" containerName="revoke-keys"
Jul 22 19:28:30.343344 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.343321    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd9de2d-32e7-49b5-b4b6-63f3f8b41c9c" containerName="revoke-keys"
Jul 22 19:28:30.343344 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.343334    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5d4c7a1-6a52-4c42-aa1d-6894c96cb5b9" containerName="maas-api"
Jul 22 19:28:30.346576 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.346557    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.349185 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.349148    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5308e856-serving-cert\""
Jul 22 19:28:30.349376 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.349356    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 19:28:30.350263 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.350245    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 19:28:30.350389 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.350251    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65h52\""
Jul 22 19:28:30.356808 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.356786    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"]
Jul 22 19:28:30.472839 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.472808    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/337030ef-48b4-4e89-a416-2e4383f2f043-openshift-service-ca\") pod \"maas-api-e2e-mt-5308e856-76f96458d4-9n2rk\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.473021 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.472922    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48xdm\" (UniqueName: \"kubernetes.io/projected/337030ef-48b4-4e89-a416-2e4383f2f043-kube-api-access-48xdm\") pod \"maas-api-e2e-mt-5308e856-76f96458d4-9n2rk\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.473021 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.472977    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/337030ef-48b4-4e89-a416-2e4383f2f043-maas-api-tls\") pod \"maas-api-e2e-mt-5308e856-76f96458d4-9n2rk\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.574083 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.574051    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/337030ef-48b4-4e89-a416-2e4383f2f043-openshift-service-ca\") pod \"maas-api-e2e-mt-5308e856-76f96458d4-9n2rk\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.574269 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.574110    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xdm\" (UniqueName: \"kubernetes.io/projected/337030ef-48b4-4e89-a416-2e4383f2f043-kube-api-access-48xdm\") pod \"maas-api-e2e-mt-5308e856-76f96458d4-9n2rk\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.574269 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.574154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/337030ef-48b4-4e89-a416-2e4383f2f043-maas-api-tls\") pod \"maas-api-e2e-mt-5308e856-76f96458d4-9n2rk\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.574670 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.574647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/337030ef-48b4-4e89-a416-2e4383f2f043-openshift-service-ca\") pod \"maas-api-e2e-mt-5308e856-76f96458d4-9n2rk\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.576577 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.576553    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/337030ef-48b4-4e89-a416-2e4383f2f043-maas-api-tls\") pod \"maas-api-e2e-mt-5308e856-76f96458d4-9n2rk\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.581970 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.581945    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48xdm\" (UniqueName: \"kubernetes.io/projected/337030ef-48b4-4e89-a416-2e4383f2f043-kube-api-access-48xdm\") pod \"maas-api-e2e-mt-5308e856-76f96458d4-9n2rk\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.658536 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.658503    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:30.796042 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:30.795936    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"]
Jul 22 19:28:30.798326 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:28:30.798297    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337030ef_48b4_4e89_a416_2e4383f2f043.slice/crio-3a7910293c10f32224b8f53379d62a5d572f3f1a8b3a30df763289db0ad764ba WatchSource:0}: Error finding container 3a7910293c10f32224b8f53379d62a5d572f3f1a8b3a30df763289db0ad764ba: Status 404 returned error can't find the container with id 3a7910293c10f32224b8f53379d62a5d572f3f1a8b3a30df763289db0ad764ba
Jul 22 19:28:31.680817 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:31.680778    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk" event={"ID":"337030ef-48b4-4e89-a416-2e4383f2f043","Type":"ContainerStarted","Data":"06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3"}
Jul 22 19:28:31.681176 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:31.680980    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:31.681176 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:31.681028    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk" event={"ID":"337030ef-48b4-4e89-a416-2e4383f2f043","Type":"ContainerStarted","Data":"3a7910293c10f32224b8f53379d62a5d572f3f1a8b3a30df763289db0ad764ba"}
Jul 22 19:28:31.698812 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:31.698741    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk" podStartSLOduration=1.1861072990000001 podStartE2EDuration="1.698725717s" totalImagesPullingTime="512.618418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:28:31.697221706 +0000 UTC m=+2261.729239237" watchObservedRunningTime="2026-07-22 19:28:31.698725717 +0000 UTC m=+2261.730743247"
Jul 22 19:28:37.143179 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:37.143145    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:28:37.691545 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:37.691517    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:28:44.641231 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.641196    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"]
Jul 22 19:28:44.645205 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.645185    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.649154 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.649122    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5308e856\"/\"default-dockercfg-dt6qv\""
Jul 22 19:28:44.649266 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.649122    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5308e856\"/\"e2e-lifecycle-model-5308e856-kserve-self-signed-certs\""
Jul 22 19:28:44.649266 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.649202    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5308e856\"/\"openshift-service-ca.crt\""
Jul 22 19:28:44.649266 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.649176    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5308e856\"/\"kube-root-ca.crt\""
Jul 22 19:28:44.654720 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.654693    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"]
Jul 22 19:28:44.695034 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.695002    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-home\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.695273 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.695247    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-dshm\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.695406 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.695282    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e88d9a7-6612-4566-ad6c-04b43326c81f-tls-certs\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.695406 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.695360    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-model-cache\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.695406 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.695385    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nwq7\" (UniqueName: \"kubernetes.io/projected/3e88d9a7-6612-4566-ad6c-04b43326c81f-kube-api-access-6nwq7\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.695580 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.695476    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-tmp-dir\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.795976 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.795944    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-tmp-dir\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.796164 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.795992    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-home\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.796164 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.796013    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-dshm\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.796164 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.796028    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e88d9a7-6612-4566-ad6c-04b43326c81f-tls-certs\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.796164 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.796094    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-model-cache\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.796164 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.796121    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nwq7\" (UniqueName: \"kubernetes.io/projected/3e88d9a7-6612-4566-ad6c-04b43326c81f-kube-api-access-6nwq7\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.796447 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.796424    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-tmp-dir\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.796504 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.796423    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-home\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.796688 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.796668    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-model-cache\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.798592 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.798563    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-dshm\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.798772 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.798730    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e88d9a7-6612-4566-ad6c-04b43326c81f-tls-certs\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.803827 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.803806    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nwq7\" (UniqueName: \"kubernetes.io/projected/3e88d9a7-6612-4566-ad6c-04b43326c81f-kube-api-access-6nwq7\") pod \"e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") " pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:44.956672 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:44.956582    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:45.095135 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:45.094990    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"]
Jul 22 19:28:45.097271 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:28:45.097245    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e88d9a7_6612_4566_ad6c_04b43326c81f.slice/crio-fed17dacbe8eb590a15f2cb52408c05044a386c3e3e9d44d6597a58f4c749a2c WatchSource:0}: Error finding container fed17dacbe8eb590a15f2cb52408c05044a386c3e3e9d44d6597a58f4c749a2c: Status 404 returned error can't find the container with id fed17dacbe8eb590a15f2cb52408c05044a386c3e3e9d44d6597a58f4c749a2c
Jul 22 19:28:45.756736 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:45.756686    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm" event={"ID":"3e88d9a7-6612-4566-ad6c-04b43326c81f","Type":"ContainerStarted","Data":"aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42"}
Jul 22 19:28:45.757118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:45.756742    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm" event={"ID":"3e88d9a7-6612-4566-ad6c-04b43326c81f","Type":"ContainerStarted","Data":"fed17dacbe8eb590a15f2cb52408c05044a386c3e3e9d44d6597a58f4c749a2c"}
Jul 22 19:28:45.773720 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:45.773664    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm" podStartSLOduration=1.773644849 podStartE2EDuration="1.773644849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:28:45.772034612 +0000 UTC m=+2275.804052146" watchObservedRunningTime="2026-07-22 19:28:45.773644849 +0000 UTC m=+2275.805662379"
Jul 22 19:28:48.743178 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:48.743141    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:28:54.957440 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:54.957398    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:54.957868 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:54.957472    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:54.970366 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:54.970341    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:28:55.824469 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:28:55.824435    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:29:07.145668 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:07.145634    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:29:27.152813 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:27.152775    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:29:28.617124 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.617088    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"]
Jul 22 19:29:28.617586 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.617339    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk" podUID="337030ef-48b4-4e89-a416-2e4383f2f043" containerName="maas-api" containerID="cri-o://06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3" gracePeriod=30
Jul 22 19:29:28.862416 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.862389    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:29:28.903496 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.903411    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/337030ef-48b4-4e89-a416-2e4383f2f043-openshift-service-ca\") pod \"337030ef-48b4-4e89-a416-2e4383f2f043\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") "
Jul 22 19:29:28.903496 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.903459    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/337030ef-48b4-4e89-a416-2e4383f2f043-maas-api-tls\") pod \"337030ef-48b4-4e89-a416-2e4383f2f043\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") "
Jul 22 19:29:28.903730 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.903506    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48xdm\" (UniqueName: \"kubernetes.io/projected/337030ef-48b4-4e89-a416-2e4383f2f043-kube-api-access-48xdm\") pod \"337030ef-48b4-4e89-a416-2e4383f2f043\" (UID: \"337030ef-48b4-4e89-a416-2e4383f2f043\") "
Jul 22 19:29:28.903822 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.903799    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/337030ef-48b4-4e89-a416-2e4383f2f043-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "337030ef-48b4-4e89-a416-2e4383f2f043" (UID: "337030ef-48b4-4e89-a416-2e4383f2f043"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:29:28.903893 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.903877    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/337030ef-48b4-4e89-a416-2e4383f2f043-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:29:28.905666 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.905639    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/337030ef-48b4-4e89-a416-2e4383f2f043-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "337030ef-48b4-4e89-a416-2e4383f2f043" (UID: "337030ef-48b4-4e89-a416-2e4383f2f043"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:29:28.905802 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.905698    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337030ef-48b4-4e89-a416-2e4383f2f043-kube-api-access-48xdm" (OuterVolumeSpecName: "kube-api-access-48xdm") pod "337030ef-48b4-4e89-a416-2e4383f2f043" (UID: "337030ef-48b4-4e89-a416-2e4383f2f043"). InnerVolumeSpecName "kube-api-access-48xdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:29:28.985061 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.985010    2558 generic.go:356] "Generic (PLEG): container finished" podID="337030ef-48b4-4e89-a416-2e4383f2f043" containerID="06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3" exitCode=0
Jul 22 19:29:28.985061 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.985062    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk" event={"ID":"337030ef-48b4-4e89-a416-2e4383f2f043","Type":"ContainerDied","Data":"06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3"}
Jul 22 19:29:28.985318 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.985067    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"
Jul 22 19:29:28.985318 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.985112    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk" event={"ID":"337030ef-48b4-4e89-a416-2e4383f2f043","Type":"ContainerDied","Data":"3a7910293c10f32224b8f53379d62a5d572f3f1a8b3a30df763289db0ad764ba"}
Jul 22 19:29:28.985318 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.985135    2558 scope.go:117] "RemoveContainer" containerID="06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3"
Jul 22 19:29:28.993915 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.993896    2558 scope.go:117] "RemoveContainer" containerID="06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3"
Jul 22 19:29:28.994171 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:29:28.994147    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3\": container with ID starting with 06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3 not found: ID does not exist" containerID="06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3"
Jul 22 19:29:28.994252 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:28.994178    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3"} err="failed to get container status \"06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3\": rpc error: code = NotFound desc = could not find container \"06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3\": container with ID starting with 06c33a5afa8e4c0298ce3c22ef12d7ce2350ac2e72661ea5ed4cd2811c08bef3 not found: ID does not exist"
Jul 22 19:29:29.004420 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:29.004399    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48xdm\" (UniqueName: \"kubernetes.io/projected/337030ef-48b4-4e89-a416-2e4383f2f043-kube-api-access-48xdm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:29:29.004506 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:29.004422    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/337030ef-48b4-4e89-a416-2e4383f2f043-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:29:29.007926 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:29.007905    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"]
Jul 22 19:29:29.011618 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:29.011593    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5308e856-76f96458d4-9n2rk"]
Jul 22 19:29:30.457542 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:30.457505    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="337030ef-48b4-4e89-a416-2e4383f2f043" path="/var/lib/kubelet/pods/337030ef-48b4-4e89-a416-2e4383f2f043/volumes"
Jul 22 19:29:37.937154 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:37.937110    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:29:40.432514 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:40.432477    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"]
Jul 22 19:29:50.553451 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:50.553375    2558 scope.go:117] "RemoveContainer" containerID="879a5abe7f1f9b799b18ee2b30d9f6ab82030b0c15805f334d609d5da59b7daf"
Jul 22 19:29:55.453363 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.453296    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm" podUID="3e88d9a7-6612-4566-ad6c-04b43326c81f" containerName="main" containerID="cri-o://aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42" gracePeriod=45
Jul 22 19:29:55.704130 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.704073    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:29:55.847924 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.847889    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-tmp-dir\") pod \"3e88d9a7-6612-4566-ad6c-04b43326c81f\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") "
Jul 22 19:29:55.848118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.847942    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-model-cache\") pod \"3e88d9a7-6612-4566-ad6c-04b43326c81f\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") "
Jul 22 19:29:55.848118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.847976    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nwq7\" (UniqueName: \"kubernetes.io/projected/3e88d9a7-6612-4566-ad6c-04b43326c81f-kube-api-access-6nwq7\") pod \"3e88d9a7-6612-4566-ad6c-04b43326c81f\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") "
Jul 22 19:29:55.848118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.848038    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-home\") pod \"3e88d9a7-6612-4566-ad6c-04b43326c81f\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") "
Jul 22 19:29:55.848118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.848056    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e88d9a7-6612-4566-ad6c-04b43326c81f-tls-certs\") pod \"3e88d9a7-6612-4566-ad6c-04b43326c81f\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") "
Jul 22 19:29:55.848118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.848093    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-dshm\") pod \"3e88d9a7-6612-4566-ad6c-04b43326c81f\" (UID: \"3e88d9a7-6612-4566-ad6c-04b43326c81f\") "
Jul 22 19:29:55.848366 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.848257    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3e88d9a7-6612-4566-ad6c-04b43326c81f" (UID: "3e88d9a7-6612-4566-ad6c-04b43326c81f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:29:55.848366 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.848301    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-model-cache" (OuterVolumeSpecName: "model-cache") pod "3e88d9a7-6612-4566-ad6c-04b43326c81f" (UID: "3e88d9a7-6612-4566-ad6c-04b43326c81f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:29:55.848366 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.848313    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-home" (OuterVolumeSpecName: "home") pod "3e88d9a7-6612-4566-ad6c-04b43326c81f" (UID: "3e88d9a7-6612-4566-ad6c-04b43326c81f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:29:55.850380 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.850352    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e88d9a7-6612-4566-ad6c-04b43326c81f-kube-api-access-6nwq7" (OuterVolumeSpecName: "kube-api-access-6nwq7") pod "3e88d9a7-6612-4566-ad6c-04b43326c81f" (UID: "3e88d9a7-6612-4566-ad6c-04b43326c81f"). InnerVolumeSpecName "kube-api-access-6nwq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:29:55.850486 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.850386    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-dshm" (OuterVolumeSpecName: "dshm") pod "3e88d9a7-6612-4566-ad6c-04b43326c81f" (UID: "3e88d9a7-6612-4566-ad6c-04b43326c81f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:29:55.850486 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.850422    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e88d9a7-6612-4566-ad6c-04b43326c81f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e88d9a7-6612-4566-ad6c-04b43326c81f" (UID: "3e88d9a7-6612-4566-ad6c-04b43326c81f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:29:55.949455 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.949423    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:29:55.949455 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.949455    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e88d9a7-6612-4566-ad6c-04b43326c81f-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:29:55.949663 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.949468    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:29:55.949663 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.949480    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:29:55.949663 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.949491    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e88d9a7-6612-4566-ad6c-04b43326c81f-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:29:55.949663 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:55.949503    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nwq7\" (UniqueName: \"kubernetes.io/projected/3e88d9a7-6612-4566-ad6c-04b43326c81f-kube-api-access-6nwq7\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:29:56.128446 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.128410    2558 generic.go:356] "Generic (PLEG): container finished" podID="3e88d9a7-6612-4566-ad6c-04b43326c81f" containerID="aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42" exitCode=0
Jul 22 19:29:56.128625 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.128454    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"
Jul 22 19:29:56.128625 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.128500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm" event={"ID":"3e88d9a7-6612-4566-ad6c-04b43326c81f","Type":"ContainerDied","Data":"aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42"}
Jul 22 19:29:56.128625 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.128542    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm" event={"ID":"3e88d9a7-6612-4566-ad6c-04b43326c81f","Type":"ContainerDied","Data":"fed17dacbe8eb590a15f2cb52408c05044a386c3e3e9d44d6597a58f4c749a2c"}
Jul 22 19:29:56.128625 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.128561    2558 scope.go:117] "RemoveContainer" containerID="aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42"
Jul 22 19:29:56.137570 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.137555    2558 scope.go:117] "RemoveContainer" containerID="aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42"
Jul 22 19:29:56.137836 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:29:56.137814    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42\": container with ID starting with aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42 not found: ID does not exist" containerID="aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42"
Jul 22 19:29:56.137934 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.137842    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42"} err="failed to get container status \"aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42\": rpc error: code = NotFound desc = could not find container \"aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42\": container with ID starting with aa8245669b030e6ae2f6e57a4326a7856085f0f5187c87d691f4b1e67df32d42 not found: ID does not exist"
Jul 22 19:29:56.155456 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.155435    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"]
Jul 22 19:29:56.158633 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.158612    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5308e856/e2e-lifecycle-model-5308e856-kserve-559cbb485f-rkdxm"]
Jul 22 19:29:56.451818 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:29:56.451704    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e88d9a7-6612-4566-ad6c-04b43326c81f" path="/var/lib/kubelet/pods/3e88d9a7-6612-4566-ad6c-04b43326c81f/volumes"
Jul 22 19:30:00.133503 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.133467    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq"]
Jul 22 19:30:00.133902 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.133871    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="337030ef-48b4-4e89-a416-2e4383f2f043" containerName="maas-api"
Jul 22 19:30:00.133902 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.133882    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="337030ef-48b4-4e89-a416-2e4383f2f043" containerName="maas-api"
Jul 22 19:30:00.133902 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.133902    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e88d9a7-6612-4566-ad6c-04b43326c81f" containerName="main"
Jul 22 19:30:00.134002 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.133907    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e88d9a7-6612-4566-ad6c-04b43326c81f" containerName="main"
Jul 22 19:30:00.134002 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.133972    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="337030ef-48b4-4e89-a416-2e4383f2f043" containerName="maas-api"
Jul 22 19:30:00.134002 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.133980    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e88d9a7-6612-4566-ad6c-04b43326c81f" containerName="main"
Jul 22 19:30:00.136950 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.136932    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq"
Jul 22 19:30:00.139232 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.139215    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\""
Jul 22 19:30:00.142735 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.142714    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq"]
Jul 22 19:30:00.148820 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.148796    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq"
Jul 22 19:30:00.283159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:00.283138    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq"]
Jul 22 19:30:00.285015 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:30:00.284991    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818b9371_2c61_4f2e_b733_ad6517ea528b.slice/crio-54b4ae55f588e84c4a6f5a3642dc8fd32f885c5910e451efe3e29876e502203c WatchSource:0}: Error finding container 54b4ae55f588e84c4a6f5a3642dc8fd32f885c5910e451efe3e29876e502203c: Status 404 returned error can't find the container with id 54b4ae55f588e84c4a6f5a3642dc8fd32f885c5910e451efe3e29876e502203c
Jul 22 19:30:01.156939 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:01.156899    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq" event={"ID":"818b9371-2c61-4f2e-b733-ad6517ea528b","Type":"ContainerStarted","Data":"98b3782b40c7eac0d5a7a95608bb66c1af3a7951e2db7b672e83bf660c3c4ac4"}
Jul 22 19:30:01.157312 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:01.156962    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq" event={"ID":"818b9371-2c61-4f2e-b733-ad6517ea528b","Type":"ContainerStarted","Data":"54b4ae55f588e84c4a6f5a3642dc8fd32f885c5910e451efe3e29876e502203c"}
Jul 22 19:30:01.172237 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:01.172179    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq" podStartSLOduration=1.172162732 podStartE2EDuration="1.172162732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:30:01.171110344 +0000 UTC m=+2351.203127874" watchObservedRunningTime="2026-07-22 19:30:01.172162732 +0000 UTC m=+2351.204180261"
Jul 22 19:30:02.163234 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:02.163201    2558 generic.go:356] "Generic (PLEG): container finished" podID="818b9371-2c61-4f2e-b733-ad6517ea528b" containerID="98b3782b40c7eac0d5a7a95608bb66c1af3a7951e2db7b672e83bf660c3c4ac4" exitCode=0
Jul 22 19:30:02.163621 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:02.163262    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq" event={"ID":"818b9371-2c61-4f2e-b733-ad6517ea528b","Type":"ContainerDied","Data":"98b3782b40c7eac0d5a7a95608bb66c1af3a7951e2db7b672e83bf660c3c4ac4"}
Jul 22 19:30:03.306075 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:03.306045    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq"
Jul 22 19:30:04.174494 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:04.174462    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq" event={"ID":"818b9371-2c61-4f2e-b733-ad6517ea528b","Type":"ContainerDied","Data":"54b4ae55f588e84c4a6f5a3642dc8fd32f885c5910e451efe3e29876e502203c"}
Jul 22 19:30:04.174494 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:04.174486    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745810-2k8nq"
Jul 22 19:30:04.174711 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:04.174511    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54b4ae55f588e84c4a6f5a3642dc8fd32f885c5910e451efe3e29876e502203c"
Jul 22 19:30:27.040551 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:27.040514    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:30:38.844370 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:38.844331    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:30:50.578135 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:30:50.578093    2558 scope.go:117] "RemoveContainer" containerID="1e24c1e3885631305c68163b4fcb789d30f8c8f6ee9af9a209074e77506fb7c3"
Jul 22 19:31:20.266171 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.266080    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"]
Jul 22 19:31:20.266737 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.266667    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818b9371-2c61-4f2e-b733-ad6517ea528b" containerName="cleanup"
Jul 22 19:31:20.266737 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.266687    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="818b9371-2c61-4f2e-b733-ad6517ea528b" containerName="cleanup"
Jul 22 19:31:20.266907 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.266833    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="818b9371-2c61-4f2e-b733-ad6517ea528b" containerName="cleanup"
Jul 22 19:31:20.269666 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.269643    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.272054 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.271997    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9c16c7-serving-cert\""
Jul 22 19:31:20.272054 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.272006    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65h52\""
Jul 22 19:31:20.273050 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.273033    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 19:31:20.273127 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.273033    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 19:31:20.279978 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.279954    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"]
Jul 22 19:31:20.309174 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.309144    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83572166-9256-4feb-931b-0ff84d20f255-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.309334 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.309193    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwvpf\" (UniqueName: \"kubernetes.io/projected/83572166-9256-4feb-931b-0ff84d20f255-kube-api-access-wwvpf\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.309334 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.309241    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83572166-9256-4feb-931b-0ff84d20f255-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.410423 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.410380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83572166-9256-4feb-931b-0ff84d20f255-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.410599 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.410477    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83572166-9256-4feb-931b-0ff84d20f255-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.410599 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.410508    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwvpf\" (UniqueName: \"kubernetes.io/projected/83572166-9256-4feb-931b-0ff84d20f255-kube-api-access-wwvpf\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.410677 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:31:20.410618    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-serving-cert: secret "maas-api-e2e-inf-a-9c16c7-serving-cert" not found
Jul 22 19:31:20.410711 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:31:20.410693    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83572166-9256-4feb-931b-0ff84d20f255-maas-api-tls podName:83572166-9256-4feb-931b-0ff84d20f255 nodeName:}" failed. No retries permitted until 2026-07-22 19:31:20.910673978 +0000 UTC m=+2430.942691486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/83572166-9256-4feb-931b-0ff84d20f255-maas-api-tls") pod "maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5" (UID: "83572166-9256-4feb-931b-0ff84d20f255") : secret "maas-api-e2e-inf-a-9c16c7-serving-cert" not found
Jul 22 19:31:20.411097 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.411079    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83572166-9256-4feb-931b-0ff84d20f255-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.422090 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.422065    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwvpf\" (UniqueName: \"kubernetes.io/projected/83572166-9256-4feb-931b-0ff84d20f255-kube-api-access-wwvpf\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.916771 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.916714    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83572166-9256-4feb-931b-0ff84d20f255-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:20.919227 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:20.919207    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83572166-9256-4feb-931b-0ff84d20f255-maas-api-tls\") pod \"maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:21.181598 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:21.181511    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:21.320116 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:21.320086    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"]
Jul 22 19:31:21.324442 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:21.324419    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 19:31:21.578837 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:21.578726    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5" event={"ID":"83572166-9256-4feb-931b-0ff84d20f255","Type":"ContainerStarted","Data":"a378011c94461caffed898f24fa100463d753f14575bae8fdfb1a0a818190e38"}
Jul 22 19:31:22.585487 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:22.585438    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5" event={"ID":"83572166-9256-4feb-931b-0ff84d20f255","Type":"ContainerStarted","Data":"9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b"}
Jul 22 19:31:22.585995 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:22.585573    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:22.601808 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:22.601729    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5" podStartSLOduration=2.179799407 podStartE2EDuration="2.601714078s" totalImagesPullingTime="421.914671ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:31:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:31:22.600865069 +0000 UTC m=+2432.632882600" watchObservedRunningTime="2026-07-22 19:31:22.601714078 +0000 UTC m=+2432.633731586"
Jul 22 19:31:27.439980 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:27.439937    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:31:28.328261 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.328226    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"]
Jul 22 19:31:28.331991 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.331965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.334391 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.334369    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-85c487-serving-cert\""
Jul 22 19:31:28.343306 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.343280    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"]
Jul 22 19:31:28.389254 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.389217    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8432d10-1663-49f4-87b5-e7fab8393212-maas-api-tls\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.389440 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.389267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8432d10-1663-49f4-87b5-e7fab8393212-openshift-service-ca\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.389440 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.389372    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-668hg\" (UniqueName: \"kubernetes.io/projected/c8432d10-1663-49f4-87b5-e7fab8393212-kube-api-access-668hg\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.490109 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.490064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8432d10-1663-49f4-87b5-e7fab8393212-openshift-service-ca\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.490590 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.490167    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-668hg\" (UniqueName: \"kubernetes.io/projected/c8432d10-1663-49f4-87b5-e7fab8393212-kube-api-access-668hg\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.490590 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.490266    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8432d10-1663-49f4-87b5-e7fab8393212-maas-api-tls\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.490590 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:31:28.490371    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-serving-cert: secret "maas-api-e2e-inf-b-85c487-serving-cert" not found
Jul 22 19:31:28.490590 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:31:28.490447    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8432d10-1663-49f4-87b5-e7fab8393212-maas-api-tls podName:c8432d10-1663-49f4-87b5-e7fab8393212 nodeName:}" failed. No retries permitted until 2026-07-22 19:31:28.990424948 +0000 UTC m=+2439.022442472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8432d10-1663-49f4-87b5-e7fab8393212-maas-api-tls") pod "maas-api-e2e-inf-b-85c487-855856c56-lfl7z" (UID: "c8432d10-1663-49f4-87b5-e7fab8393212") : secret "maas-api-e2e-inf-b-85c487-serving-cert" not found
Jul 22 19:31:28.490932 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.490909    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8432d10-1663-49f4-87b5-e7fab8393212-openshift-service-ca\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.506006 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.505979    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-668hg\" (UniqueName: \"kubernetes.io/projected/c8432d10-1663-49f4-87b5-e7fab8393212-kube-api-access-668hg\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.598034 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.597942    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:31:28.994017 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.993974    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8432d10-1663-49f4-87b5-e7fab8393212-maas-api-tls\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:28.996373 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:28.996346    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8432d10-1663-49f4-87b5-e7fab8393212-maas-api-tls\") pod \"maas-api-e2e-inf-b-85c487-855856c56-lfl7z\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:29.244726 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:29.244634    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:29.586572 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:29.586537    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"]
Jul 22 19:31:29.588482 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:31:29.588445    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8432d10_1663_49f4_87b5_e7fab8393212.slice/crio-bf49dd2385c8e5a44ed83a43e5e024a4ca65136cf843c41ea6f589ac63c98e3d WatchSource:0}: Error finding container bf49dd2385c8e5a44ed83a43e5e024a4ca65136cf843c41ea6f589ac63c98e3d: Status 404 returned error can't find the container with id bf49dd2385c8e5a44ed83a43e5e024a4ca65136cf843c41ea6f589ac63c98e3d
Jul 22 19:31:29.624491 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:29.624452    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z" event={"ID":"c8432d10-1663-49f4-87b5-e7fab8393212","Type":"ContainerStarted","Data":"bf49dd2385c8e5a44ed83a43e5e024a4ca65136cf843c41ea6f589ac63c98e3d"}
Jul 22 19:31:30.636420 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:30.636369    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z" event={"ID":"c8432d10-1663-49f4-87b5-e7fab8393212","Type":"ContainerStarted","Data":"afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637"}
Jul 22 19:31:30.636938 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:30.636575    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:30.653729 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:30.653688    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z" podStartSLOduration=1.937402626 podStartE2EDuration="2.653675039s" totalImagesPullingTime="716.272413ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:31:30.652966917 +0000 UTC m=+2440.684984446" watchObservedRunningTime="2026-07-22 19:31:30.653675039 +0000 UTC m=+2440.685692569"
Jul 22 19:31:36.648487 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:36.648460    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:31:38.344319 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:38.344280    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:31:40.028860 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.028819    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"]
Jul 22 19:31:40.033096 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.033077    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.036763 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.036728    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9c16c7\"/\"default-dockercfg-rmr2s\""
Jul 22 19:31:40.036763 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.036728    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9c16c7\"/\"test-model-9c16c7-kserve-self-signed-certs\""
Jul 22 19:31:40.036971 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.036771    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9c16c7\"/\"openshift-service-ca.crt\""
Jul 22 19:31:40.036971 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.036730    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9c16c7\"/\"kube-root-ca.crt\""
Jul 22 19:31:40.042623 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.042604    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"]
Jul 22 19:31:40.195473 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.195442    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gq7d\" (UniqueName: \"kubernetes.io/projected/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-kube-api-access-5gq7d\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.195473 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.195483    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-home\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.195701 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.195518    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tmp-dir\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.195701 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.195567    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-model-cache\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.195701 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.195585    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-dshm\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.195701 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.195625    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tls-certs\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.296876 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.296787    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gq7d\" (UniqueName: \"kubernetes.io/projected/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-kube-api-access-5gq7d\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.296876 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.296844    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-home\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.297118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.296890    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tmp-dir\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.297118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.296929    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-model-cache\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.297118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.296954    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-dshm\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.297118 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.296990    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tls-certs\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.297370 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.297348    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-home\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.297442 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.297377    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tmp-dir\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.297442 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.297420    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-model-cache\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.299281 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.299254    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-dshm\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.299588 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.299569    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tls-certs\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.305194 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.305169    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gq7d\" (UniqueName: \"kubernetes.io/projected/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-kube-api-access-5gq7d\") pod \"test-model-9c16c7-kserve-6cd57b96dc-jcjgd\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") " pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.343179 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.343156    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:40.475878 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.475860    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"]
Jul 22 19:31:40.692490 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.692456    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd" event={"ID":"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa","Type":"ContainerStarted","Data":"53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149"}
Jul 22 19:31:40.692684 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.692500    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd" event={"ID":"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa","Type":"ContainerStarted","Data":"42990aee3ec6bcf588f7a1e05eb2be0bb973c043c8f0282ba29e297e43741cad"}
Jul 22 19:31:40.708129 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:40.708071    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd" podStartSLOduration=0.708052763 podStartE2EDuration="708.052763ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:31:40.707933524 +0000 UTC m=+2450.739951054" watchObservedRunningTime="2026-07-22 19:31:40.708052763 +0000 UTC m=+2450.740070295"
Jul 22 19:31:50.241076 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:50.241038    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:31:50.344075 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:50.344046    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:50.344333 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:50.344308    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:50.356618 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:50.356592    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:31:50.605278 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:50.605255    2558 scope.go:117] "RemoveContainer" containerID="c11e9ffad959f85212f5995684bfa2b96c68099425f77fde12003be368dc3552"
Jul 22 19:31:50.613872 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:50.613852    2558 scope.go:117] "RemoveContainer" containerID="839207347b36ab52fd8e0e681c4acef90a9a5270e944a594a87ac58f6883f912"
Jul 22 19:31:50.756020 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:31:50.755990    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:32:04.941471 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:04.941437    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"]
Jul 22 19:32:04.945086 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:04.945060    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:04.948868 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:04.948846    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-85c487\"/\"default-dockercfg-7f2gh\""
Jul 22 19:32:04.948988 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:04.948921    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-85c487\"/\"test-model-85c487-kserve-self-signed-certs\""
Jul 22 19:32:04.949084 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:04.949060    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-85c487\"/\"kube-root-ca.crt\""
Jul 22 19:32:04.949194 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:04.949138    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-85c487\"/\"openshift-service-ca.crt\""
Jul 22 19:32:04.954332 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:04.954303    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"]
Jul 22 19:32:05.015509 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.015058    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tls-certs\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.015509 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.015126    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7mwm\" (UniqueName: \"kubernetes.io/projected/7a2714c1-c748-47cd-8411-e65fa6dbbe65-kube-api-access-p7mwm\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.015509 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.015209    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-dshm\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.015509 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.015243    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-model-cache\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.015509 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.015263    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tmp-dir\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.015509 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.015325    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-home\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.116876 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.116843    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tls-certs\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.117090 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.116884    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7mwm\" (UniqueName: \"kubernetes.io/projected/7a2714c1-c748-47cd-8411-e65fa6dbbe65-kube-api-access-p7mwm\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.117090 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.116933    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-dshm\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.117090 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.116948    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-model-cache\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.117090 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.116966    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tmp-dir\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.117090 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.117006    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-home\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.117561 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.117536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-home\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.117662 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.117536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-model-cache\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.117662 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.117582    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tmp-dir\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.119277 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.119252    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-dshm\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.119845 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.119824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tls-certs\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.124528 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.124499    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7mwm\" (UniqueName: \"kubernetes.io/projected/7a2714c1-c748-47cd-8411-e65fa6dbbe65-kube-api-access-p7mwm\") pod \"test-model-85c487-kserve-b6957b8c-r5fd2\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") " pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.255314 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.255220    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:05.406038 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.405921    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"]
Jul 22 19:32:05.408956 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:32:05.408930    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a2714c1_c748_47cd_8411_e65fa6dbbe65.slice/crio-a7a50ff059f80f2d03588f5074964776c86c34908cd1c0d6a7ec234a37e69f64 WatchSource:0}: Error finding container a7a50ff059f80f2d03588f5074964776c86c34908cd1c0d6a7ec234a37e69f64: Status 404 returned error can't find the container with id a7a50ff059f80f2d03588f5074964776c86c34908cd1c0d6a7ec234a37e69f64
Jul 22 19:32:05.821566 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.821531    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2" event={"ID":"7a2714c1-c748-47cd-8411-e65fa6dbbe65","Type":"ContainerStarted","Data":"ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf"}
Jul 22 19:32:05.821734 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.821574    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2" event={"ID":"7a2714c1-c748-47cd-8411-e65fa6dbbe65","Type":"ContainerStarted","Data":"a7a50ff059f80f2d03588f5074964776c86c34908cd1c0d6a7ec234a37e69f64"}
Jul 22 19:32:05.841098 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:05.841039    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2" podStartSLOduration=1.84102292 podStartE2EDuration="1.84102292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:32:05.838625394 +0000 UTC m=+2475.870642924" watchObservedRunningTime="2026-07-22 19:32:05.84102292 +0000 UTC m=+2475.873040483"
Jul 22 19:32:07.446773 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:07.446721    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:32:15.256260 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:15.256211    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:15.256782 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:15.256282    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:15.268872 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:15.268851    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:15.883432 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:15.883399    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:32:24.635886 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:24.635845    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:32:47.079094 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:47.079014    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"]
Jul 22 19:32:48.052528 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:48.052490    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"]
Jul 22 19:32:53.939168 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:53.939126    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:32:58.556251 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.556211    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"]
Jul 22 19:32:58.557062 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.557015    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5" podUID="83572166-9256-4feb-931b-0ff84d20f255" containerName="maas-api" containerID="cri-o://9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b" gracePeriod=30
Jul 22 19:32:58.592321 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.592283    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5" podUID="83572166-9256-4feb-931b-0ff84d20f255" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.56:8443/health\": dial tcp 10.132.0.56:8443: connect: connection refused"
Jul 22 19:32:58.814242 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.814182    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:32:58.911531 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.911496    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83572166-9256-4feb-931b-0ff84d20f255-openshift-service-ca\") pod \"83572166-9256-4feb-931b-0ff84d20f255\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") "
Jul 22 19:32:58.911733 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.911595    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwvpf\" (UniqueName: \"kubernetes.io/projected/83572166-9256-4feb-931b-0ff84d20f255-kube-api-access-wwvpf\") pod \"83572166-9256-4feb-931b-0ff84d20f255\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") "
Jul 22 19:32:58.911733 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.911720    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83572166-9256-4feb-931b-0ff84d20f255-maas-api-tls\") pod \"83572166-9256-4feb-931b-0ff84d20f255\" (UID: \"83572166-9256-4feb-931b-0ff84d20f255\") "
Jul 22 19:32:58.911981 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.911956    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83572166-9256-4feb-931b-0ff84d20f255-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83572166-9256-4feb-931b-0ff84d20f255" (UID: "83572166-9256-4feb-931b-0ff84d20f255"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:32:58.912081 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.912060    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83572166-9256-4feb-931b-0ff84d20f255-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:32:58.913678 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.913656    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83572166-9256-4feb-931b-0ff84d20f255-kube-api-access-wwvpf" (OuterVolumeSpecName: "kube-api-access-wwvpf") pod "83572166-9256-4feb-931b-0ff84d20f255" (UID: "83572166-9256-4feb-931b-0ff84d20f255"). InnerVolumeSpecName "kube-api-access-wwvpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:32:58.913767 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:58.913686    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83572166-9256-4feb-931b-0ff84d20f255-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83572166-9256-4feb-931b-0ff84d20f255" (UID: "83572166-9256-4feb-931b-0ff84d20f255"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:32:59.012533 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.012498    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83572166-9256-4feb-931b-0ff84d20f255-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:32:59.012533 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.012528    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwvpf\" (UniqueName: \"kubernetes.io/projected/83572166-9256-4feb-931b-0ff84d20f255-kube-api-access-wwvpf\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:32:59.103957 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.103920    2558 generic.go:356] "Generic (PLEG): container finished" podID="83572166-9256-4feb-931b-0ff84d20f255" containerID="9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b" exitCode=0
Jul 22 19:32:59.104145 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.103968    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"
Jul 22 19:32:59.104145 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.104011    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5" event={"ID":"83572166-9256-4feb-931b-0ff84d20f255","Type":"ContainerDied","Data":"9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b"}
Jul 22 19:32:59.104145 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.104072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5" event={"ID":"83572166-9256-4feb-931b-0ff84d20f255","Type":"ContainerDied","Data":"a378011c94461caffed898f24fa100463d753f14575bae8fdfb1a0a818190e38"}
Jul 22 19:32:59.104145 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.104104    2558 scope.go:117] "RemoveContainer" containerID="9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b"
Jul 22 19:32:59.114147 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.114128    2558 scope.go:117] "RemoveContainer" containerID="9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b"
Jul 22 19:32:59.114424 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:32:59.114406    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b\": container with ID starting with 9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b not found: ID does not exist" containerID="9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b"
Jul 22 19:32:59.114471 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.114433    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b"} err="failed to get container status \"9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b\": rpc error: code = NotFound desc = could not find container \"9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b\": container with ID starting with 9bf62fb174663bdcd2168f5973e1d891cb0d975350009025d5a311078212284b not found: ID does not exist"
Jul 22 19:32:59.125954 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.125923    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"]
Jul 22 19:32:59.129411 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:32:59.129389    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9c16c7-79d668cb79-994h5"]
Jul 22 19:33:00.451697 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:00.451661    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83572166-9256-4feb-931b-0ff84d20f255" path="/var/lib/kubelet/pods/83572166-9256-4feb-931b-0ff84d20f255/volumes"
Jul 22 19:33:02.100063 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.100023    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd" podUID="f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" containerName="main" containerID="cri-o://53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149" gracePeriod=45
Jul 22 19:33:02.349457 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.349433    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:33:02.444502 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.444413    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gq7d\" (UniqueName: \"kubernetes.io/projected/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-kube-api-access-5gq7d\") pod \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") "
Jul 22 19:33:02.444502 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.444454    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-dshm\") pod \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") "
Jul 22 19:33:02.444502 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.444478    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tls-certs\") pod \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") "
Jul 22 19:33:02.444835 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.444541    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-home\") pod \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") "
Jul 22 19:33:02.444835 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.444588    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-model-cache\") pod \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") "
Jul 22 19:33:02.444835 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.444636    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tmp-dir\") pod \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\" (UID: \"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa\") "
Jul 22 19:33:02.445012 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.444902    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" (UID: "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:33:02.445012 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.444947    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" (UID: "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:33:02.445012 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.444910    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-home" (OuterVolumeSpecName: "home") pod "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" (UID: "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:33:02.446810 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.446779    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-kube-api-access-5gq7d" (OuterVolumeSpecName: "kube-api-access-5gq7d") pod "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" (UID: "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa"). InnerVolumeSpecName "kube-api-access-5gq7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:33:02.446810 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.446788    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" (UID: "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:33:02.446989 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.446923    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-dshm" (OuterVolumeSpecName: "dshm") pod "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" (UID: "f13e54d9-6d1a-44c8-aec1-7b1a756f08aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:33:02.545550 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.545509    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:02.545550 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.545542    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:02.545550 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.545555    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gq7d\" (UniqueName: \"kubernetes.io/projected/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-kube-api-access-5gq7d\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:02.545832 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.545568    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:02.545832 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.545583    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:02.545832 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:02.545594    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:03.073490 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.073442    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2" podUID="7a2714c1-c748-47cd-8411-e65fa6dbbe65" containerName="main" containerID="cri-o://ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf" gracePeriod=45
Jul 22 19:33:03.132733 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.132703    2558 generic.go:356] "Generic (PLEG): container finished" podID="f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" containerID="53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149" exitCode=0
Jul 22 19:33:03.133115 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.132738    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd" event={"ID":"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa","Type":"ContainerDied","Data":"53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149"}
Jul 22 19:33:03.133115 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.132778    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd" event={"ID":"f13e54d9-6d1a-44c8-aec1-7b1a756f08aa","Type":"ContainerDied","Data":"42990aee3ec6bcf588f7a1e05eb2be0bb973c043c8f0282ba29e297e43741cad"}
Jul 22 19:33:03.133115 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.132785    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"
Jul 22 19:33:03.133115 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.132797    2558 scope.go:117] "RemoveContainer" containerID="53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149"
Jul 22 19:33:03.148136 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.148111    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"]
Jul 22 19:33:03.152127 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.152105    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-9c16c7/test-model-9c16c7-kserve-6cd57b96dc-jcjgd"]
Jul 22 19:33:03.184603 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.184577    2558 scope.go:117] "RemoveContainer" containerID="53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149"
Jul 22 19:33:03.185007 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:33:03.184979    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149\": container with ID starting with 53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149 not found: ID does not exist" containerID="53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149"
Jul 22 19:33:03.185099 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.185019    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149"} err="failed to get container status \"53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149\": rpc error: code = NotFound desc = could not find container \"53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149\": container with ID starting with 53ba49e6540bf5bbe514327667fcc7d5aade64f91f95766913dfd40baf376149 not found: ID does not exist"
Jul 22 19:33:03.333543 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.333474    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:33:03.454256 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.454227    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7mwm\" (UniqueName: \"kubernetes.io/projected/7a2714c1-c748-47cd-8411-e65fa6dbbe65-kube-api-access-p7mwm\") pod \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") "
Jul 22 19:33:03.454256 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.454262    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-model-cache\") pod \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") "
Jul 22 19:33:03.454481 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.454328    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-dshm\") pod \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") "
Jul 22 19:33:03.454481 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.454348    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-home\") pod \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") "
Jul 22 19:33:03.454481 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.454369    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tmp-dir\") pod \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") "
Jul 22 19:33:03.454481 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.454443    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tls-certs\") pod \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\" (UID: \"7a2714c1-c748-47cd-8411-e65fa6dbbe65\") "
Jul 22 19:33:03.454688 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.454633    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a2714c1-c748-47cd-8411-e65fa6dbbe65" (UID: "7a2714c1-c748-47cd-8411-e65fa6dbbe65"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:33:03.454688 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.454651    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-home" (OuterVolumeSpecName: "home") pod "7a2714c1-c748-47cd-8411-e65fa6dbbe65" (UID: "7a2714c1-c748-47cd-8411-e65fa6dbbe65"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:33:03.454831 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.454722    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a2714c1-c748-47cd-8411-e65fa6dbbe65" (UID: "7a2714c1-c748-47cd-8411-e65fa6dbbe65"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:33:03.456557 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.456537    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a2714c1-c748-47cd-8411-e65fa6dbbe65-kube-api-access-p7mwm" (OuterVolumeSpecName: "kube-api-access-p7mwm") pod "7a2714c1-c748-47cd-8411-e65fa6dbbe65" (UID: "7a2714c1-c748-47cd-8411-e65fa6dbbe65"). InnerVolumeSpecName "kube-api-access-p7mwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:33:03.456638 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.456537    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a2714c1-c748-47cd-8411-e65fa6dbbe65" (UID: "7a2714c1-c748-47cd-8411-e65fa6dbbe65"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:33:03.456638 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.456587    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-dshm" (OuterVolumeSpecName: "dshm") pod "7a2714c1-c748-47cd-8411-e65fa6dbbe65" (UID: "7a2714c1-c748-47cd-8411-e65fa6dbbe65"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:33:03.555543 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.555507    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:03.555543 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.555537    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7mwm\" (UniqueName: \"kubernetes.io/projected/7a2714c1-c748-47cd-8411-e65fa6dbbe65-kube-api-access-p7mwm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:03.555543 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.555547    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:03.555543 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.555555    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:03.555841 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.555562    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:03.555841 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.555590    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a2714c1-c748-47cd-8411-e65fa6dbbe65-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:03.642335 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:03.642294    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:33:04.139828 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.139795    2558 generic.go:356] "Generic (PLEG): container finished" podID="7a2714c1-c748-47cd-8411-e65fa6dbbe65" containerID="ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf" exitCode=0
Jul 22 19:33:04.140306 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.139841    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"
Jul 22 19:33:04.140306 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.139888    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2" event={"ID":"7a2714c1-c748-47cd-8411-e65fa6dbbe65","Type":"ContainerDied","Data":"ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf"}
Jul 22 19:33:04.140306 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.139935    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2" event={"ID":"7a2714c1-c748-47cd-8411-e65fa6dbbe65","Type":"ContainerDied","Data":"a7a50ff059f80f2d03588f5074964776c86c34908cd1c0d6a7ec234a37e69f64"}
Jul 22 19:33:04.140306 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.139961    2558 scope.go:117] "RemoveContainer" containerID="ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf"
Jul 22 19:33:04.149053 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.149036    2558 scope.go:117] "RemoveContainer" containerID="ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf"
Jul 22 19:33:04.149292 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:33:04.149274    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf\": container with ID starting with ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf not found: ID does not exist" containerID="ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf"
Jul 22 19:33:04.149360 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.149306    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf"} err="failed to get container status \"ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf\": rpc error: code = NotFound desc = could not find container \"ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf\": container with ID starting with ac46c3473f5e887c7dc2e46777aa91bbfcf34fdda8aba5f3f9cbc95af4ed76cf not found: ID does not exist"
Jul 22 19:33:04.160086 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.160062    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"]
Jul 22 19:33:04.163526 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.163507    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-85c487/test-model-85c487-kserve-b6957b8c-r5fd2"]
Jul 22 19:33:04.452134 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.452057    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a2714c1-c748-47cd-8411-e65fa6dbbe65" path="/var/lib/kubelet/pods/7a2714c1-c748-47cd-8411-e65fa6dbbe65/volumes"
Jul 22 19:33:04.452512 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:04.452497    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" path="/var/lib/kubelet/pods/f13e54d9-6d1a-44c8-aec1-7b1a756f08aa/volumes"
Jul 22 19:33:20.400420 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.400382    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"]
Jul 22 19:33:20.401276 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.401212    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z" podUID="c8432d10-1663-49f4-87b5-e7fab8393212" containerName="maas-api" containerID="cri-o://afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637" gracePeriod=30
Jul 22 19:33:20.651089 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.651017    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:33:20.809072 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.809032    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-668hg\" (UniqueName: \"kubernetes.io/projected/c8432d10-1663-49f4-87b5-e7fab8393212-kube-api-access-668hg\") pod \"c8432d10-1663-49f4-87b5-e7fab8393212\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") "
Jul 22 19:33:20.809278 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.809095    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8432d10-1663-49f4-87b5-e7fab8393212-openshift-service-ca\") pod \"c8432d10-1663-49f4-87b5-e7fab8393212\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") "
Jul 22 19:33:20.809278 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.809132    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8432d10-1663-49f4-87b5-e7fab8393212-maas-api-tls\") pod \"c8432d10-1663-49f4-87b5-e7fab8393212\" (UID: \"c8432d10-1663-49f4-87b5-e7fab8393212\") "
Jul 22 19:33:20.809540 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.809510    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8432d10-1663-49f4-87b5-e7fab8393212-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8432d10-1663-49f4-87b5-e7fab8393212" (UID: "c8432d10-1663-49f4-87b5-e7fab8393212"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:33:20.811190 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.811166    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8432d10-1663-49f4-87b5-e7fab8393212-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8432d10-1663-49f4-87b5-e7fab8393212" (UID: "c8432d10-1663-49f4-87b5-e7fab8393212"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:33:20.811272 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.811200    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8432d10-1663-49f4-87b5-e7fab8393212-kube-api-access-668hg" (OuterVolumeSpecName: "kube-api-access-668hg") pod "c8432d10-1663-49f4-87b5-e7fab8393212" (UID: "c8432d10-1663-49f4-87b5-e7fab8393212"). InnerVolumeSpecName "kube-api-access-668hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:33:20.910220 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.910130    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-668hg\" (UniqueName: \"kubernetes.io/projected/c8432d10-1663-49f4-87b5-e7fab8393212-kube-api-access-668hg\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:20.910220 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.910166    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8432d10-1663-49f4-87b5-e7fab8393212-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:20.910220 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:20.910177    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8432d10-1663-49f4-87b5-e7fab8393212-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:33:21.227735 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:21.227641    2558 generic.go:356] "Generic (PLEG): container finished" podID="c8432d10-1663-49f4-87b5-e7fab8393212" containerID="afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637" exitCode=0
Jul 22 19:33:21.227735 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:21.227697    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"
Jul 22 19:33:21.227735 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:21.227723    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z" event={"ID":"c8432d10-1663-49f4-87b5-e7fab8393212","Type":"ContainerDied","Data":"afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637"}
Jul 22 19:33:21.228034 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:21.227795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z" event={"ID":"c8432d10-1663-49f4-87b5-e7fab8393212","Type":"ContainerDied","Data":"bf49dd2385c8e5a44ed83a43e5e024a4ca65136cf843c41ea6f589ac63c98e3d"}
Jul 22 19:33:21.228034 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:21.227819    2558 scope.go:117] "RemoveContainer" containerID="afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637"
Jul 22 19:33:21.236848 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:21.236827    2558 scope.go:117] "RemoveContainer" containerID="afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637"
Jul 22 19:33:21.237117 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:33:21.237097    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637\": container with ID starting with afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637 not found: ID does not exist" containerID="afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637"
Jul 22 19:33:21.237196 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:21.237124    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637"} err="failed to get container status \"afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637\": rpc error: code = NotFound desc = could not find container \"afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637\": container with ID starting with afab3631e5afc9ad6a3220026baaafd7b738fa47abd14eb9b2e4cd2f492bc637 not found: ID does not exist"
Jul 22 19:33:21.248046 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:21.248022    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"]
Jul 22 19:33:21.251987 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:21.251966    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-85c487-855856c56-lfl7z"]
Jul 22 19:33:22.451514 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:22.451485    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8432d10-1663-49f4-87b5-e7fab8393212" path="/var/lib/kubelet/pods/c8432d10-1663-49f4-87b5-e7fab8393212/volumes"
Jul 22 19:33:23.333857 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:23.333821    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:33:35.061824 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.061789    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"]
Jul 22 19:33:35.062208 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062191    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a2714c1-c748-47cd-8411-e65fa6dbbe65" containerName="main"
Jul 22 19:33:35.062208 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062202    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a2714c1-c748-47cd-8411-e65fa6dbbe65" containerName="main"
Jul 22 19:33:35.062284 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062216    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8432d10-1663-49f4-87b5-e7fab8393212" containerName="maas-api"
Jul 22 19:33:35.062284 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062221    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8432d10-1663-49f4-87b5-e7fab8393212" containerName="maas-api"
Jul 22 19:33:35.062284 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062236    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83572166-9256-4feb-931b-0ff84d20f255" containerName="maas-api"
Jul 22 19:33:35.062284 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062245    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="83572166-9256-4feb-931b-0ff84d20f255" containerName="maas-api"
Jul 22 19:33:35.062284 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062254    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" containerName="main"
Jul 22 19:33:35.062284 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062259    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" containerName="main"
Jul 22 19:33:35.062468 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062322    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8432d10-1663-49f4-87b5-e7fab8393212" containerName="maas-api"
Jul 22 19:33:35.062468 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062335    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13e54d9-6d1a-44c8-aec1-7b1a756f08aa" containerName="main"
Jul 22 19:33:35.062468 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062341    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a2714c1-c748-47cd-8411-e65fa6dbbe65" containerName="main"
Jul 22 19:33:35.062468 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.062348    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="83572166-9256-4feb-931b-0ff84d20f255" containerName="maas-api"
Jul 22 19:33:35.065342 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.065324    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.067763 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.067713    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-802be2-serving-cert\""
Jul 22 19:33:35.067867 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.067838    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 19:33:35.067929 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.067907    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65h52\""
Jul 22 19:33:35.068666 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.068649    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 19:33:35.072665 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.072645    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"]
Jul 22 19:33:35.130075 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.130041    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af467b2a-f2fe-434b-9048-e3c033b8772e-maas-api-tls\") pod \"maas-api-e2e-api-a-802be2-669675dd8d-vcht7\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.130268 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.130090    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25df9\" (UniqueName: \"kubernetes.io/projected/af467b2a-f2fe-434b-9048-e3c033b8772e-kube-api-access-25df9\") pod \"maas-api-e2e-api-a-802be2-669675dd8d-vcht7\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.130268 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.130169    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af467b2a-f2fe-434b-9048-e3c033b8772e-openshift-service-ca\") pod \"maas-api-e2e-api-a-802be2-669675dd8d-vcht7\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.231346 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.231307    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af467b2a-f2fe-434b-9048-e3c033b8772e-openshift-service-ca\") pod \"maas-api-e2e-api-a-802be2-669675dd8d-vcht7\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.231526 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.231389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af467b2a-f2fe-434b-9048-e3c033b8772e-maas-api-tls\") pod \"maas-api-e2e-api-a-802be2-669675dd8d-vcht7\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.231569 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.231522    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25df9\" (UniqueName: \"kubernetes.io/projected/af467b2a-f2fe-434b-9048-e3c033b8772e-kube-api-access-25df9\") pod \"maas-api-e2e-api-a-802be2-669675dd8d-vcht7\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.232136 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.232110    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af467b2a-f2fe-434b-9048-e3c033b8772e-openshift-service-ca\") pod \"maas-api-e2e-api-a-802be2-669675dd8d-vcht7\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.233971 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.233947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af467b2a-f2fe-434b-9048-e3c033b8772e-maas-api-tls\") pod \"maas-api-e2e-api-a-802be2-669675dd8d-vcht7\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.239495 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.239475    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25df9\" (UniqueName: \"kubernetes.io/projected/af467b2a-f2fe-434b-9048-e3c033b8772e-kube-api-access-25df9\") pod \"maas-api-e2e-api-a-802be2-669675dd8d-vcht7\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.377559 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.377527    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:35.521002 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:35.520827    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"]
Jul 22 19:33:35.523309 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:33:35.523279    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf467b2a_f2fe_434b_9048_e3c033b8772e.slice/crio-aa8ec38feed0bdb64bec598ed967eaeaff064fbf489ade90809421d6c01dd3bb WatchSource:0}: Error finding container aa8ec38feed0bdb64bec598ed967eaeaff064fbf489ade90809421d6c01dd3bb: Status 404 returned error can't find the container with id aa8ec38feed0bdb64bec598ed967eaeaff064fbf489ade90809421d6c01dd3bb
Jul 22 19:33:36.307968 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:36.307924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7" event={"ID":"af467b2a-f2fe-434b-9048-e3c033b8772e","Type":"ContainerStarted","Data":"3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31"}
Jul 22 19:33:36.308339 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:36.308020    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7" event={"ID":"af467b2a-f2fe-434b-9048-e3c033b8772e","Type":"ContainerStarted","Data":"aa8ec38feed0bdb64bec598ed967eaeaff064fbf489ade90809421d6c01dd3bb"}
Jul 22 19:33:36.308339 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:36.308051    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:36.325063 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:36.325013    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7" podStartSLOduration=0.743076621 podStartE2EDuration="1.324998778s" totalImagesPullingTime="581.922157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:33:36.322995274 +0000 UTC m=+2566.355012804" watchObservedRunningTime="2026-07-22 19:33:36.324998778 +0000 UTC m=+2566.357016360"
Jul 22 19:33:42.318135 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:42.318105    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:33:42.938724 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:42.938692    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:33:43.115883 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.115853    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"]
Jul 22 19:33:43.119826 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.119803    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.122883 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.122857    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-61193d-serving-cert\""
Jul 22 19:33:43.132714 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.132691    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"]
Jul 22 19:33:43.202130 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.202048    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90e57a80-b1c7-49c5-bf24-f2e136ed975c-maas-api-tls\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.202130 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.202086    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90e57a80-b1c7-49c5-bf24-f2e136ed975c-openshift-service-ca\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.202130 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.202106    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4lwr\" (UniqueName: \"kubernetes.io/projected/90e57a80-b1c7-49c5-bf24-f2e136ed975c-kube-api-access-k4lwr\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.303545 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.303510    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90e57a80-b1c7-49c5-bf24-f2e136ed975c-maas-api-tls\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.303545 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.303546    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90e57a80-b1c7-49c5-bf24-f2e136ed975c-openshift-service-ca\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.303845 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.303571    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4lwr\" (UniqueName: \"kubernetes.io/projected/90e57a80-b1c7-49c5-bf24-f2e136ed975c-kube-api-access-k4lwr\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.303845 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:33:43.303675    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-serving-cert: secret "maas-api-e2e-api-b-61193d-serving-cert" not found
Jul 22 19:33:43.303845 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:33:43.303782    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e57a80-b1c7-49c5-bf24-f2e136ed975c-maas-api-tls podName:90e57a80-b1c7-49c5-bf24-f2e136ed975c nodeName:}" failed. No retries permitted until 2026-07-22 19:33:43.803727865 +0000 UTC m=+2573.835745372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/90e57a80-b1c7-49c5-bf24-f2e136ed975c-maas-api-tls") pod "maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8" (UID: "90e57a80-b1c7-49c5-bf24-f2e136ed975c") : secret "maas-api-e2e-api-b-61193d-serving-cert" not found
Jul 22 19:33:43.304193 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.304174    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90e57a80-b1c7-49c5-bf24-f2e136ed975c-openshift-service-ca\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.312738 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.312715    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4lwr\" (UniqueName: \"kubernetes.io/projected/90e57a80-b1c7-49c5-bf24-f2e136ed975c-kube-api-access-k4lwr\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.807727 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.807694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90e57a80-b1c7-49c5-bf24-f2e136ed975c-maas-api-tls\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:43.809987 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:43.809962    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90e57a80-b1c7-49c5-bf24-f2e136ed975c-maas-api-tls\") pod \"maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:44.033681 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:44.033645    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:44.167295 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:44.167270    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"]
Jul 22 19:33:44.169798 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:33:44.169765    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e57a80_b1c7_49c5_bf24_f2e136ed975c.slice/crio-d84be225650296fbbb8781266f661f53144140564049611d1a6e5a813ab44c3e WatchSource:0}: Error finding container d84be225650296fbbb8781266f661f53144140564049611d1a6e5a813ab44c3e: Status 404 returned error can't find the container with id d84be225650296fbbb8781266f661f53144140564049611d1a6e5a813ab44c3e
Jul 22 19:33:44.349400 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:44.349310    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8" event={"ID":"90e57a80-b1c7-49c5-bf24-f2e136ed975c","Type":"ContainerStarted","Data":"d84be225650296fbbb8781266f661f53144140564049611d1a6e5a813ab44c3e"}
Jul 22 19:33:45.355435 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:45.355398    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8" event={"ID":"90e57a80-b1c7-49c5-bf24-f2e136ed975c","Type":"ContainerStarted","Data":"b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a"}
Jul 22 19:33:45.355858 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:45.355475    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:45.372386 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:45.372340    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8" podStartSLOduration=1.673654612 podStartE2EDuration="2.372320611s" totalImagesPullingTime="698.665999ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:33:45.371341384 +0000 UTC m=+2575.403358934" watchObservedRunningTime="2026-07-22 19:33:45.372320611 +0000 UTC m=+2575.404338123"
Jul 22 19:33:50.644015 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:50.643977    2558 scope.go:117] "RemoveContainer" containerID="186f485dcb37089b1e63cbfa3c22d94ebf55cbc3f43cb3c39a1ab86ce837c144"
Jul 22 19:33:51.367082 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:51.367053    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:33:58.839079 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:33:58.839043    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:34:11.386029 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.385998    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"]
Jul 22 19:34:11.386542 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.386235    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7" podUID="af467b2a-f2fe-434b-9048-e3c033b8772e" containerName="maas-api" containerID="cri-o://3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31" gracePeriod=30
Jul 22 19:34:11.635764 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.635719    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:34:11.764271 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.764174    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af467b2a-f2fe-434b-9048-e3c033b8772e-maas-api-tls\") pod \"af467b2a-f2fe-434b-9048-e3c033b8772e\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") "
Jul 22 19:34:11.764271 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.764247    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25df9\" (UniqueName: \"kubernetes.io/projected/af467b2a-f2fe-434b-9048-e3c033b8772e-kube-api-access-25df9\") pod \"af467b2a-f2fe-434b-9048-e3c033b8772e\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") "
Jul 22 19:34:11.764518 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.764304    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af467b2a-f2fe-434b-9048-e3c033b8772e-openshift-service-ca\") pod \"af467b2a-f2fe-434b-9048-e3c033b8772e\" (UID: \"af467b2a-f2fe-434b-9048-e3c033b8772e\") "
Jul 22 19:34:11.764674 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.764648    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af467b2a-f2fe-434b-9048-e3c033b8772e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af467b2a-f2fe-434b-9048-e3c033b8772e" (UID: "af467b2a-f2fe-434b-9048-e3c033b8772e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:34:11.766310 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.766275    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af467b2a-f2fe-434b-9048-e3c033b8772e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af467b2a-f2fe-434b-9048-e3c033b8772e" (UID: "af467b2a-f2fe-434b-9048-e3c033b8772e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:34:11.766310 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.766293    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af467b2a-f2fe-434b-9048-e3c033b8772e-kube-api-access-25df9" (OuterVolumeSpecName: "kube-api-access-25df9") pod "af467b2a-f2fe-434b-9048-e3c033b8772e" (UID: "af467b2a-f2fe-434b-9048-e3c033b8772e"). InnerVolumeSpecName "kube-api-access-25df9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:34:11.865820 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.865742    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af467b2a-f2fe-434b-9048-e3c033b8772e-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:34:11.865820 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.865809    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25df9\" (UniqueName: \"kubernetes.io/projected/af467b2a-f2fe-434b-9048-e3c033b8772e-kube-api-access-25df9\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:34:11.865820 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:11.865819    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af467b2a-f2fe-434b-9048-e3c033b8772e-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:34:12.491568 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:12.491534    2558 generic.go:356] "Generic (PLEG): container finished" podID="af467b2a-f2fe-434b-9048-e3c033b8772e" containerID="3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31" exitCode=0
Jul 22 19:34:12.491964 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:12.491581    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"
Jul 22 19:34:12.491964 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:12.491618    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7" event={"ID":"af467b2a-f2fe-434b-9048-e3c033b8772e","Type":"ContainerDied","Data":"3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31"}
Jul 22 19:34:12.491964 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:12.491674    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7" event={"ID":"af467b2a-f2fe-434b-9048-e3c033b8772e","Type":"ContainerDied","Data":"aa8ec38feed0bdb64bec598ed967eaeaff064fbf489ade90809421d6c01dd3bb"}
Jul 22 19:34:12.491964 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:12.491701    2558 scope.go:117] "RemoveContainer" containerID="3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31"
Jul 22 19:34:12.500660 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:12.500642    2558 scope.go:117] "RemoveContainer" containerID="3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31"
Jul 22 19:34:12.500980 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:34:12.500953    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31\": container with ID starting with 3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31 not found: ID does not exist" containerID="3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31"
Jul 22 19:34:12.501053 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:12.500989    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31"} err="failed to get container status \"3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31\": rpc error: code = NotFound desc = could not find container \"3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31\": container with ID starting with 3586dc7054c9ea2b4d8405d4a57237a2ee4f753297f6d1f56207c9c9acdf7e31 not found: ID does not exist"
Jul 22 19:34:12.508096 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:12.508073    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"]
Jul 22 19:34:12.511169 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:12.511149    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-802be2-669675dd8d-vcht7"]
Jul 22 19:34:14.451439 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:14.451406    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af467b2a-f2fe-434b-9048-e3c033b8772e" path="/var/lib/kubelet/pods/af467b2a-f2fe-434b-9048-e3c033b8772e/volumes"
Jul 22 19:34:15.341189 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:15.341153    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:34:22.230241 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.230199    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"]
Jul 22 19:34:22.230843 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.230823    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af467b2a-f2fe-434b-9048-e3c033b8772e" containerName="maas-api"
Jul 22 19:34:22.230963 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.230845    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="af467b2a-f2fe-434b-9048-e3c033b8772e" containerName="maas-api"
Jul 22 19:34:22.230963 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.230942    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="af467b2a-f2fe-434b-9048-e3c033b8772e" containerName="maas-api"
Jul 22 19:34:22.235582 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.235561    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"
Jul 22 19:34:22.238480 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.238457    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\""
Jul 22 19:34:22.239708 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.239676    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"]
Jul 22 19:34:22.251292 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.251267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fe390a-fa79-4eb5-83b3-0d0d29c248cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-61193d-x4fvp\" (UID: \"67fe390a-fa79-4eb5-83b3-0d0d29c248cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"
Jul 22 19:34:22.351867 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.351832    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fe390a-fa79-4eb5-83b3-0d0d29c248cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-61193d-x4fvp\" (UID: \"67fe390a-fa79-4eb5-83b3-0d0d29c248cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"
Jul 22 19:34:22.352478 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.352458    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fe390a-fa79-4eb5-83b3-0d0d29c248cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-61193d-x4fvp\" (UID: \"67fe390a-fa79-4eb5-83b3-0d0d29c248cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"
Jul 22 19:34:22.547292 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.547207    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"
Jul 22 19:34:22.884855 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:22.884827    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"]
Jul 22 19:34:22.886998 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:34:22.886970    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67fe390a_fa79_4eb5_83b3_0d0d29c248cd.slice/crio-050c40e9de97c3437b75aed337fb31fe776e03241f469e940d64ae675952cd3a WatchSource:0}: Error finding container 050c40e9de97c3437b75aed337fb31fe776e03241f469e940d64ae675952cd3a: Status 404 returned error can't find the container with id 050c40e9de97c3437b75aed337fb31fe776e03241f469e940d64ae675952cd3a
Jul 22 19:34:23.558478 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:23.558383    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp" event={"ID":"67fe390a-fa79-4eb5-83b3-0d0d29c248cd","Type":"ContainerStarted","Data":"d95321960d94d4c6246f270271c9837956ef288e6a5a7135d7ab238bdb907f26"}
Jul 22 19:34:23.558478 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:23.558436    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp" event={"ID":"67fe390a-fa79-4eb5-83b3-0d0d29c248cd","Type":"ContainerStarted","Data":"050c40e9de97c3437b75aed337fb31fe776e03241f469e940d64ae675952cd3a"}
Jul 22 19:34:23.573798 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:23.573732    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp" podStartSLOduration=1.5737175479999999 podStartE2EDuration="1.573717548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:34:23.571810285 +0000 UTC m=+2613.603827797" watchObservedRunningTime="2026-07-22 19:34:23.573717548 +0000 UTC m=+2613.605735124"
Jul 22 19:34:24.564305 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:24.564269    2558 generic.go:356] "Generic (PLEG): container finished" podID="67fe390a-fa79-4eb5-83b3-0d0d29c248cd" containerID="d95321960d94d4c6246f270271c9837956ef288e6a5a7135d7ab238bdb907f26" exitCode=0
Jul 22 19:34:24.564716 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:24.564362    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp" event={"ID":"67fe390a-fa79-4eb5-83b3-0d0d29c248cd","Type":"ContainerDied","Data":"d95321960d94d4c6246f270271c9837956ef288e6a5a7135d7ab238bdb907f26"}
Jul 22 19:34:25.704506 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:25.704480    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"
Jul 22 19:34:25.781595 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:25.781565    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fe390a-fa79-4eb5-83b3-0d0d29c248cd-maas-api-service-ca\") pod \"67fe390a-fa79-4eb5-83b3-0d0d29c248cd\" (UID: \"67fe390a-fa79-4eb5-83b3-0d0d29c248cd\") "
Jul 22 19:34:25.781924 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:25.781904    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67fe390a-fa79-4eb5-83b3-0d0d29c248cd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "67fe390a-fa79-4eb5-83b3-0d0d29c248cd" (UID: "67fe390a-fa79-4eb5-83b3-0d0d29c248cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:34:25.882771 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:25.882715    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fe390a-fa79-4eb5-83b3-0d0d29c248cd-maas-api-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:34:26.575044 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:26.575013    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"
Jul 22 19:34:26.575044 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:26.575040    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp" event={"ID":"67fe390a-fa79-4eb5-83b3-0d0d29c248cd","Type":"ContainerDied","Data":"050c40e9de97c3437b75aed337fb31fe776e03241f469e940d64ae675952cd3a"}
Jul 22 19:34:26.575254 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:26.575081    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="050c40e9de97c3437b75aed337fb31fe776e03241f469e940d64ae675952cd3a"
Jul 22 19:34:37.732072 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:37.732025    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"]
Jul 22 19:34:37.732614 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:37.732287    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8" podUID="90e57a80-b1c7-49c5-bf24-f2e136ed975c" containerName="maas-api" containerID="cri-o://b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a" gracePeriod=30
Jul 22 19:34:37.987983 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:37.987913    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:34:38.093340 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.093302    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90e57a80-b1c7-49c5-bf24-f2e136ed975c-openshift-service-ca\") pod \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") "
Jul 22 19:34:38.093545 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.093408    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90e57a80-b1c7-49c5-bf24-f2e136ed975c-maas-api-tls\") pod \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") "
Jul 22 19:34:38.093622 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.093566    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4lwr\" (UniqueName: \"kubernetes.io/projected/90e57a80-b1c7-49c5-bf24-f2e136ed975c-kube-api-access-k4lwr\") pod \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\" (UID: \"90e57a80-b1c7-49c5-bf24-f2e136ed975c\") "
Jul 22 19:34:38.093805 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.093737    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90e57a80-b1c7-49c5-bf24-f2e136ed975c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "90e57a80-b1c7-49c5-bf24-f2e136ed975c" (UID: "90e57a80-b1c7-49c5-bf24-f2e136ed975c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:34:38.093894 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.093879    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90e57a80-b1c7-49c5-bf24-f2e136ed975c-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:34:38.095483 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.095457    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90e57a80-b1c7-49c5-bf24-f2e136ed975c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "90e57a80-b1c7-49c5-bf24-f2e136ed975c" (UID: "90e57a80-b1c7-49c5-bf24-f2e136ed975c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:34:38.095631 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.095615    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90e57a80-b1c7-49c5-bf24-f2e136ed975c-kube-api-access-k4lwr" (OuterVolumeSpecName: "kube-api-access-k4lwr") pod "90e57a80-b1c7-49c5-bf24-f2e136ed975c" (UID: "90e57a80-b1c7-49c5-bf24-f2e136ed975c"). InnerVolumeSpecName "kube-api-access-k4lwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:34:38.194731 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.194693    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90e57a80-b1c7-49c5-bf24-f2e136ed975c-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:34:38.194731 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.194722    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4lwr\" (UniqueName: \"kubernetes.io/projected/90e57a80-b1c7-49c5-bf24-f2e136ed975c-kube-api-access-k4lwr\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:34:38.638225 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.638186    2558 generic.go:356] "Generic (PLEG): container finished" podID="90e57a80-b1c7-49c5-bf24-f2e136ed975c" containerID="b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a" exitCode=0
Jul 22 19:34:38.638394 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.638236    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"
Jul 22 19:34:38.638394 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.638272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8" event={"ID":"90e57a80-b1c7-49c5-bf24-f2e136ed975c","Type":"ContainerDied","Data":"b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a"}
Jul 22 19:34:38.638394 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.638328    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8" event={"ID":"90e57a80-b1c7-49c5-bf24-f2e136ed975c","Type":"ContainerDied","Data":"d84be225650296fbbb8781266f661f53144140564049611d1a6e5a813ab44c3e"}
Jul 22 19:34:38.638394 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.638350    2558 scope.go:117] "RemoveContainer" containerID="b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a"
Jul 22 19:34:38.647032 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.647008    2558 scope.go:117] "RemoveContainer" containerID="b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a"
Jul 22 19:34:38.647288 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:34:38.647269    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a\": container with ID starting with b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a not found: ID does not exist" containerID="b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a"
Jul 22 19:34:38.647336 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.647297    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a"} err="failed to get container status \"b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a\": rpc error: code = NotFound desc = could not find container \"b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a\": container with ID starting with b51b8d5c93c132c35a27b170f0b2d5a5cbf2f05cb9145bae506435c35ef9332a not found: ID does not exist"
Jul 22 19:34:38.654773 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.654736    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"]
Jul 22 19:34:38.659306 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:38.659278    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-61193d-654b4c6cfb-gknj8"]
Jul 22 19:34:40.451729 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:40.451696    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90e57a80-b1c7-49c5-bf24-f2e136ed975c" path="/var/lib/kubelet/pods/90e57a80-b1c7-49c5-bf24-f2e136ed975c/volumes"
Jul 22 19:34:42.234033 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:42.233995    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:34:42.306920 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:42.306878    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"]
Jul 22 19:34:42.309168 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:42.309142    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-61193d-x4fvp"]
Jul 22 19:34:42.451319 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:42.451288    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67fe390a-fa79-4eb5-83b3-0d0d29c248cd" path="/var/lib/kubelet/pods/67fe390a-fa79-4eb5-83b3-0d0d29c248cd/volumes"
Jul 22 19:34:52.561658 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.561620    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"]
Jul 22 19:34:52.562052 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.562018    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90e57a80-b1c7-49c5-bf24-f2e136ed975c" containerName="maas-api"
Jul 22 19:34:52.562052 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.562030    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="90e57a80-b1c7-49c5-bf24-f2e136ed975c" containerName="maas-api"
Jul 22 19:34:52.562052 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.562050    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67fe390a-fa79-4eb5-83b3-0d0d29c248cd" containerName="revoke-keys"
Jul 22 19:34:52.562171 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.562056    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="67fe390a-fa79-4eb5-83b3-0d0d29c248cd" containerName="revoke-keys"
Jul 22 19:34:52.562171 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.562114    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="90e57a80-b1c7-49c5-bf24-f2e136ed975c" containerName="maas-api"
Jul 22 19:34:52.562171 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.562123    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="67fe390a-fa79-4eb5-83b3-0d0d29c248cd" containerName="revoke-keys"
Jul 22 19:34:52.566532 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.566514    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:52.569283 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.569261    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-488695-serving-cert\""
Jul 22 19:34:52.569390 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.569271    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 19:34:52.570403 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.570386    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65h52\""
Jul 22 19:34:52.570501 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.570417    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 19:34:52.574555 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.574459    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"]
Jul 22 19:34:52.717461 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.717426    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:52.717636 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.717475    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-maas-api-tls\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:52.717636 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.717527    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blg48\" (UniqueName: \"kubernetes.io/projected/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-kube-api-access-blg48\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:52.818894 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.818802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:52.818894 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.818863    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-maas-api-tls\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:52.819124 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:34:52.818945    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-serving-cert: secret "maas-api-e2e-shared-a-488695-serving-cert" not found
Jul 22 19:34:52.819124 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.818969    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blg48\" (UniqueName: \"kubernetes.io/projected/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-kube-api-access-blg48\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:52.819124 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:34:52.819004    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-maas-api-tls podName:c50f1e73-6b64-49dd-aedc-cae9cd4671ed nodeName:}" failed. No retries permitted until 2026-07-22 19:34:53.318987663 +0000 UTC m=+2643.351005171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-maas-api-tls") pod "maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq" (UID: "c50f1e73-6b64-49dd-aedc-cae9cd4671ed") : secret "maas-api-e2e-shared-a-488695-serving-cert" not found
Jul 22 19:34:52.819536 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.819511    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:52.827093 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:52.827068    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blg48\" (UniqueName: \"kubernetes.io/projected/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-kube-api-access-blg48\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:53.322742 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:53.322705    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-maas-api-tls\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:53.325048 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:53.325024    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-maas-api-tls\") pod \"maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:53.477710 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:53.477671    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:53.624360 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:53.624332    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"]
Jul 22 19:34:53.626293 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:34:53.626268    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc50f1e73_6b64_49dd_aedc_cae9cd4671ed.slice/crio-e7de5f8e03228b985da84e3c4d19c1951fe879c59ba5d1a3da5d07461a34ea46 WatchSource:0}: Error finding container e7de5f8e03228b985da84e3c4d19c1951fe879c59ba5d1a3da5d07461a34ea46: Status 404 returned error can't find the container with id e7de5f8e03228b985da84e3c4d19c1951fe879c59ba5d1a3da5d07461a34ea46
Jul 22 19:34:53.713472 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:53.713439    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq" event={"ID":"c50f1e73-6b64-49dd-aedc-cae9cd4671ed","Type":"ContainerStarted","Data":"e7de5f8e03228b985da84e3c4d19c1951fe879c59ba5d1a3da5d07461a34ea46"}
Jul 22 19:34:54.719257 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:54.719226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq" event={"ID":"c50f1e73-6b64-49dd-aedc-cae9cd4671ed","Type":"ContainerStarted","Data":"ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4"}
Jul 22 19:34:54.719662 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:54.719333    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:34:54.742643 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:54.742595    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq" podStartSLOduration=2.257866445 podStartE2EDuration="2.742579646s" totalImagesPullingTime="484.713201ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:34:54.740639652 +0000 UTC m=+2644.772657183" watchObservedRunningTime="2026-07-22 19:34:54.742579646 +0000 UTC m=+2644.774597175"
Jul 22 19:34:59.935496 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:34:59.935462    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:35:00.575562 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.575524    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"]
Jul 22 19:35:00.579238 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.579215    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.581661 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.581641    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-f3fe27-serving-cert\""
Jul 22 19:35:00.589981 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.589954    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"]
Jul 22 19:35:00.693444 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.693413    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjq7g\" (UniqueName: \"kubernetes.io/projected/b63e7611-93b2-495b-ade6-0c897848000d-kube-api-access-vjq7g\") pod \"maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.693640 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.693554    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b63e7611-93b2-495b-ade6-0c897848000d-maas-api-tls\") pod \"maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.693640 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.693582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b63e7611-93b2-495b-ade6-0c897848000d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.729980 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.729952    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:35:00.794485 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.794450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b63e7611-93b2-495b-ade6-0c897848000d-maas-api-tls\") pod \"maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.794485 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.794482    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b63e7611-93b2-495b-ade6-0c897848000d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.794730 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.794513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjq7g\" (UniqueName: \"kubernetes.io/projected/b63e7611-93b2-495b-ade6-0c897848000d-kube-api-access-vjq7g\") pod \"maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.795191 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.795165    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b63e7611-93b2-495b-ade6-0c897848000d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.796877 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.796856    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b63e7611-93b2-495b-ade6-0c897848000d-maas-api-tls\") pod \"maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.803193 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.803173    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjq7g\" (UniqueName: \"kubernetes.io/projected/b63e7611-93b2-495b-ade6-0c897848000d-kube-api-access-vjq7g\") pod \"maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:00.891228 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:00.891200    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:01.030301 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:01.030271    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"]
Jul 22 19:35:01.031868 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:35:01.031840    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb63e7611_93b2_495b_ade6_0c897848000d.slice/crio-abba33a626726ea634bbffb1984104e2c74d49869f54bf7e4fc7917bdeb94985 WatchSource:0}: Error finding container abba33a626726ea634bbffb1984104e2c74d49869f54bf7e4fc7917bdeb94985: Status 404 returned error can't find the container with id abba33a626726ea634bbffb1984104e2c74d49869f54bf7e4fc7917bdeb94985
Jul 22 19:35:01.758070 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:01.757985    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc" event={"ID":"b63e7611-93b2-495b-ade6-0c897848000d","Type":"ContainerStarted","Data":"de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2"}
Jul 22 19:35:01.758070 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:01.758066    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc" event={"ID":"b63e7611-93b2-495b-ade6-0c897848000d","Type":"ContainerStarted","Data":"abba33a626726ea634bbffb1984104e2c74d49869f54bf7e4fc7917bdeb94985"}
Jul 22 19:35:01.758247 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:01.758102    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:01.779853 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:01.779803    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc" podStartSLOduration=1.353759789 podStartE2EDuration="1.77978217s" totalImagesPullingTime="426.022381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:35:01.779425592 +0000 UTC m=+2651.811443122" watchObservedRunningTime="2026-07-22 19:35:01.77978217 +0000 UTC m=+2651.811799700"
Jul 22 19:35:07.768397 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:07.768370    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:35:11.343661 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:11.343621    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:35:27.843461 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.843427    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"]
Jul 22 19:35:27.848468 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.848450    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.851183 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.851160    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-488695\"/\"openshift-service-ca.crt\""
Jul 22 19:35:27.851360 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.851203    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-488695\"/\"default-dockercfg-df2ff\""
Jul 22 19:35:27.851360 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.851253    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-488695\"/\"kube-root-ca.crt\""
Jul 22 19:35:27.852488 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.852468    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-488695\"/\"auth-test-model-488695-kserve-self-signed-certs\""
Jul 22 19:35:27.855735 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.855710    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"]
Jul 22 19:35:27.892897 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.892869    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tls-certs\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.892897 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.892902    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tmp-dir\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.893109 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.892919    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-dshm\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.893109 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.892936    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-model-cache\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.893109 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.893045    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4szxl\" (UniqueName: \"kubernetes.io/projected/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-kube-api-access-4szxl\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.893109 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.893107    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-home\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.994473 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.994439    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tls-certs\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.994473 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.994475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tmp-dir\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.994713 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.994495    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-dshm\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.994713 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.994607    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-model-cache\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.994713 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.994695    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szxl\" (UniqueName: \"kubernetes.io/projected/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-kube-api-access-4szxl\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.994919 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.994797    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-home\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.994919 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.994886    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tmp-dir\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.995042 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.994982    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-model-cache\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.995107 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.995082    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-home\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.996723 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.996695    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-dshm\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:27.997002 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:27.996984    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tls-certs\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:28.004960 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:28.004938    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4szxl\" (UniqueName: \"kubernetes.io/projected/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-kube-api-access-4szxl\") pod \"auth-test-model-488695-kserve-5986459b87-kvfjp\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") " pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:28.159431 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:28.159398    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:28.293558 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:28.293530    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"]
Jul 22 19:35:28.295411 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:35:28.295382    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87a1b8b4_db7c_4e23_ac09_2afde18e19ed.slice/crio-95042906d237e5f05c737736397d743e3487ce68b06d2482d7faa6e64f58db28 WatchSource:0}: Error finding container 95042906d237e5f05c737736397d743e3487ce68b06d2482d7faa6e64f58db28: Status 404 returned error can't find the container with id 95042906d237e5f05c737736397d743e3487ce68b06d2482d7faa6e64f58db28
Jul 22 19:35:28.892997 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:28.892959    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp" event={"ID":"87a1b8b4-db7c-4e23-ac09-2afde18e19ed","Type":"ContainerStarted","Data":"9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5"}
Jul 22 19:35:28.893378 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:28.893005    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp" event={"ID":"87a1b8b4-db7c-4e23-ac09-2afde18e19ed","Type":"ContainerStarted","Data":"95042906d237e5f05c737736397d743e3487ce68b06d2482d7faa6e64f58db28"}
Jul 22 19:35:28.910412 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:28.910364    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp" podStartSLOduration=1.9103490779999999 podStartE2EDuration="1.910349078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:35:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:35:28.909236223 +0000 UTC m=+2678.941253775" watchObservedRunningTime="2026-07-22 19:35:28.910349078 +0000 UTC m=+2678.942366586"
Jul 22 19:35:31.044946 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:31.044909    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:35:38.159822 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:38.159789    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:38.160231 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:38.160111    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:38.172423 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:38.172399    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:35:38.957534 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:35:38.957501    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:36:16.245653 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.245610    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"]
Jul 22 19:36:16.249673 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.249650    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.253021 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.252993    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"auth-test-model-f3fe27-kserve-self-signed-certs\""
Jul 22 19:36:16.253021 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.253005    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"openshift-service-ca.crt\""
Jul 22 19:36:16.253189 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.253000    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"kube-root-ca.crt\""
Jul 22 19:36:16.253189 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.253043    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"default-dockercfg-4d4nd\""
Jul 22 19:36:16.259399 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.259375    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"]
Jul 22 19:36:16.328427 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.328398    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbgvt\" (UniqueName: \"kubernetes.io/projected/d38a324a-c4ad-4808-8f72-aeb01bb61596-kube-api-access-qbgvt\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.328560 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.328456    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-tmp-dir\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.328560 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.328492    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-home\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.328560 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.328541    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-dshm\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.328672 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.328565    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-model-cache\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.328672 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.328588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d38a324a-c4ad-4808-8f72-aeb01bb61596-tls-certs\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.429017 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.428982    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-dshm\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.429017 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.429020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-model-cache\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.429207 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.429043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d38a324a-c4ad-4808-8f72-aeb01bb61596-tls-certs\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.429207 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.429086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbgvt\" (UniqueName: \"kubernetes.io/projected/d38a324a-c4ad-4808-8f72-aeb01bb61596-kube-api-access-qbgvt\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.429207 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.429121    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-tmp-dir\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.429207 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.429146    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-home\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.429499 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.429477    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-model-cache\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.429568 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.429532    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-home\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.429568 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.429542    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-tmp-dir\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.431267 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.431246    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-dshm\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.431526 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.431507    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d38a324a-c4ad-4808-8f72-aeb01bb61596-tls-certs\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.436213 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.436196    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbgvt\" (UniqueName: \"kubernetes.io/projected/d38a324a-c4ad-4808-8f72-aeb01bb61596-kube-api-access-qbgvt\") pod \"auth-test-model-f3fe27-kserve-5c48877575-dv4fh\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") " pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.560411 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.560345    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:16.691247 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:16.691217    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"]
Jul 22 19:36:16.693195 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:36:16.693164    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd38a324a_c4ad_4808_8f72_aeb01bb61596.slice/crio-2c13f1a175ac8d821a4dd4b862e34c297ff51ba45e031e8ea8ca290b52106fc3 WatchSource:0}: Error finding container 2c13f1a175ac8d821a4dd4b862e34c297ff51ba45e031e8ea8ca290b52106fc3: Status 404 returned error can't find the container with id 2c13f1a175ac8d821a4dd4b862e34c297ff51ba45e031e8ea8ca290b52106fc3
Jul 22 19:36:17.144701 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:17.144664    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh" event={"ID":"d38a324a-c4ad-4808-8f72-aeb01bb61596","Type":"ContainerStarted","Data":"b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e"}
Jul 22 19:36:17.144884 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:17.144711    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh" event={"ID":"d38a324a-c4ad-4808-8f72-aeb01bb61596","Type":"ContainerStarted","Data":"2c13f1a175ac8d821a4dd4b862e34c297ff51ba45e031e8ea8ca290b52106fc3"}
Jul 22 19:36:17.162078 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:17.162031    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh" podStartSLOduration=1.162017163 podStartE2EDuration="1.162017163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:36:17.160911777 +0000 UTC m=+2727.192929321" watchObservedRunningTime="2026-07-22 19:36:17.162017163 +0000 UTC m=+2727.194034692"
Jul 22 19:36:19.745138 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:19.745100    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:36:26.560773 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:26.560712    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:26.561269 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:26.560798    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:26.572829 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:26.572803    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:27.207722 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:27.207692    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:36:34.345365 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:34.345325    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:36:52.443459 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:36:52.443421    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:37:10.247638 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:37:10.247598    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:37:27.745698 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:37:27.745652    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:37:44.639271 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:37:44.639232    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:38:03.135920 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:03.135877    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:38:13.940449 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:13.940416    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"]
Jul 22 19:38:15.031430 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:15.031397    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"]
Jul 22 19:38:18.038290 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:18.038256    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:38:28.960440 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:28.960378    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp" podUID="87a1b8b4-db7c-4e23-ac09-2afde18e19ed" containerName="main" containerID="cri-o://9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5" gracePeriod=45
Jul 22 19:38:29.221013 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.220937    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:38:29.258707 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.258668    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4szxl\" (UniqueName: \"kubernetes.io/projected/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-kube-api-access-4szxl\") pod \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") "
Jul 22 19:38:29.258925 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.258710    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-home\") pod \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") "
Jul 22 19:38:29.258925 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.258857    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-dshm\") pod \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") "
Jul 22 19:38:29.258925 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.258883    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-model-cache\") pod \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") "
Jul 22 19:38:29.258925 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.258924    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tmp-dir\") pod \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") "
Jul 22 19:38:29.259145 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.258967    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tls-certs\") pod \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\" (UID: \"87a1b8b4-db7c-4e23-ac09-2afde18e19ed\") "
Jul 22 19:38:29.259145 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.259055    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-home" (OuterVolumeSpecName: "home") pod "87a1b8b4-db7c-4e23-ac09-2afde18e19ed" (UID: "87a1b8b4-db7c-4e23-ac09-2afde18e19ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:38:29.259257 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.259147    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "87a1b8b4-db7c-4e23-ac09-2afde18e19ed" (UID: "87a1b8b4-db7c-4e23-ac09-2afde18e19ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:38:29.259257 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.259235    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "87a1b8b4-db7c-4e23-ac09-2afde18e19ed" (UID: "87a1b8b4-db7c-4e23-ac09-2afde18e19ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:38:29.259433 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.259411    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:29.259503 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.259438    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:29.259503 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.259452    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:29.261156 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.261124    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-dshm" (OuterVolumeSpecName: "dshm") pod "87a1b8b4-db7c-4e23-ac09-2afde18e19ed" (UID: "87a1b8b4-db7c-4e23-ac09-2afde18e19ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:38:29.261262 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.261192    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87a1b8b4-db7c-4e23-ac09-2afde18e19ed" (UID: "87a1b8b4-db7c-4e23-ac09-2afde18e19ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:38:29.261262 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.261205    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-kube-api-access-4szxl" (OuterVolumeSpecName: "kube-api-access-4szxl") pod "87a1b8b4-db7c-4e23-ac09-2afde18e19ed" (UID: "87a1b8b4-db7c-4e23-ac09-2afde18e19ed"). InnerVolumeSpecName "kube-api-access-4szxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:38:29.360338 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.360282    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4szxl\" (UniqueName: \"kubernetes.io/projected/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-kube-api-access-4szxl\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:29.360338 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.360329    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:29.360338 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.360344    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87a1b8b4-db7c-4e23-ac09-2afde18e19ed-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:29.827227 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.827187    2558 generic.go:356] "Generic (PLEG): container finished" podID="87a1b8b4-db7c-4e23-ac09-2afde18e19ed" containerID="9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5" exitCode=0
Jul 22 19:38:29.827227 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.827234    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"
Jul 22 19:38:29.827451 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.827270    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp" event={"ID":"87a1b8b4-db7c-4e23-ac09-2afde18e19ed","Type":"ContainerDied","Data":"9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5"}
Jul 22 19:38:29.827451 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.827318    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp" event={"ID":"87a1b8b4-db7c-4e23-ac09-2afde18e19ed","Type":"ContainerDied","Data":"95042906d237e5f05c737736397d743e3487ce68b06d2482d7faa6e64f58db28"}
Jul 22 19:38:29.827451 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.827335    2558 scope.go:117] "RemoveContainer" containerID="9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5"
Jul 22 19:38:29.836531 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.836512    2558 scope.go:117] "RemoveContainer" containerID="9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5"
Jul 22 19:38:29.836804 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:38:29.836784    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5\": container with ID starting with 9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5 not found: ID does not exist" containerID="9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5"
Jul 22 19:38:29.836872 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.836813    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5"} err="failed to get container status \"9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5\": rpc error: code = NotFound desc = could not find container \"9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5\": container with ID starting with 9f52a2415e8808021fa050c3a25b0fa90afb5dd8f81d96b78277038602811bb5 not found: ID does not exist"
Jul 22 19:38:29.848140 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.848110    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"]
Jul 22 19:38:29.852035 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:29.852008    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-488695/auth-test-model-488695-kserve-5986459b87-kvfjp"]
Jul 22 19:38:30.052349 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.052305    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh" podUID="d38a324a-c4ad-4808-8f72-aeb01bb61596" containerName="main" containerID="cri-o://b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e" gracePeriod=45
Jul 22 19:38:30.312166 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.312140    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:38:30.369036 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.368994    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-tmp-dir\") pod \"d38a324a-c4ad-4808-8f72-aeb01bb61596\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") "
Jul 22 19:38:30.369036 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369040    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-home\") pod \"d38a324a-c4ad-4808-8f72-aeb01bb61596\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") "
Jul 22 19:38:30.369298 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369059    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d38a324a-c4ad-4808-8f72-aeb01bb61596-tls-certs\") pod \"d38a324a-c4ad-4808-8f72-aeb01bb61596\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") "
Jul 22 19:38:30.369298 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369079    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbgvt\" (UniqueName: \"kubernetes.io/projected/d38a324a-c4ad-4808-8f72-aeb01bb61596-kube-api-access-qbgvt\") pod \"d38a324a-c4ad-4808-8f72-aeb01bb61596\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") "
Jul 22 19:38:30.369298 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369113    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-dshm\") pod \"d38a324a-c4ad-4808-8f72-aeb01bb61596\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") "
Jul 22 19:38:30.369298 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369158    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-model-cache\") pod \"d38a324a-c4ad-4808-8f72-aeb01bb61596\" (UID: \"d38a324a-c4ad-4808-8f72-aeb01bb61596\") "
Jul 22 19:38:30.369510 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369326    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d38a324a-c4ad-4808-8f72-aeb01bb61596" (UID: "d38a324a-c4ad-4808-8f72-aeb01bb61596"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:38:30.369510 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369362    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-home" (OuterVolumeSpecName: "home") pod "d38a324a-c4ad-4808-8f72-aeb01bb61596" (UID: "d38a324a-c4ad-4808-8f72-aeb01bb61596"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:38:30.369510 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369439    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-model-cache" (OuterVolumeSpecName: "model-cache") pod "d38a324a-c4ad-4808-8f72-aeb01bb61596" (UID: "d38a324a-c4ad-4808-8f72-aeb01bb61596"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:38:30.369619 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369610    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:30.369656 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369625    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:30.369656 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.369634    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:30.371422 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.371401    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38a324a-c4ad-4808-8f72-aeb01bb61596-kube-api-access-qbgvt" (OuterVolumeSpecName: "kube-api-access-qbgvt") pod "d38a324a-c4ad-4808-8f72-aeb01bb61596" (UID: "d38a324a-c4ad-4808-8f72-aeb01bb61596"). InnerVolumeSpecName "kube-api-access-qbgvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:38:30.371622 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.371605    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-dshm" (OuterVolumeSpecName: "dshm") pod "d38a324a-c4ad-4808-8f72-aeb01bb61596" (UID: "d38a324a-c4ad-4808-8f72-aeb01bb61596"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:38:30.371682 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.371602    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d38a324a-c4ad-4808-8f72-aeb01bb61596-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d38a324a-c4ad-4808-8f72-aeb01bb61596" (UID: "d38a324a-c4ad-4808-8f72-aeb01bb61596"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:38:30.451879 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.451798    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87a1b8b4-db7c-4e23-ac09-2afde18e19ed" path="/var/lib/kubelet/pods/87a1b8b4-db7c-4e23-ac09-2afde18e19ed/volumes"
Jul 22 19:38:30.470407 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.470370    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d38a324a-c4ad-4808-8f72-aeb01bb61596-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:30.470407 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.470403    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbgvt\" (UniqueName: \"kubernetes.io/projected/d38a324a-c4ad-4808-8f72-aeb01bb61596-kube-api-access-qbgvt\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:30.470596 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.470416    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d38a324a-c4ad-4808-8f72-aeb01bb61596-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:38:30.835133 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.835049    2558 generic.go:356] "Generic (PLEG): container finished" podID="d38a324a-c4ad-4808-8f72-aeb01bb61596" containerID="b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e" exitCode=0
Jul 22 19:38:30.835133 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.835094    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"
Jul 22 19:38:30.835133 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.835124    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh" event={"ID":"d38a324a-c4ad-4808-8f72-aeb01bb61596","Type":"ContainerDied","Data":"b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e"}
Jul 22 19:38:30.835389 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.835154    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh" event={"ID":"d38a324a-c4ad-4808-8f72-aeb01bb61596","Type":"ContainerDied","Data":"2c13f1a175ac8d821a4dd4b862e34c297ff51ba45e031e8ea8ca290b52106fc3"}
Jul 22 19:38:30.835389 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.835172    2558 scope.go:117] "RemoveContainer" containerID="b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e"
Jul 22 19:38:30.843638 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.843618    2558 scope.go:117] "RemoveContainer" containerID="b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e"
Jul 22 19:38:30.843977 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:38:30.843951    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e\": container with ID starting with b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e not found: ID does not exist" containerID="b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e"
Jul 22 19:38:30.844036 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.843987    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e"} err="failed to get container status \"b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e\": rpc error: code = NotFound desc = could not find container \"b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e\": container with ID starting with b5af30f02ca06c1cc158ae0753cbd896a232e7afb261622d386d2e033561101e not found: ID does not exist"
Jul 22 19:38:30.853906 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.853879    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"]
Jul 22 19:38:30.857586 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:30.857559    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/auth-test-model-f3fe27-kserve-5c48877575-dv4fh"]
Jul 22 19:38:32.452152 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:32.452124    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38a324a-c4ad-4808-8f72-aeb01bb61596" path="/var/lib/kubelet/pods/d38a324a-c4ad-4808-8f72-aeb01bb61596/volumes"
Jul 22 19:38:59.354435 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.351400    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"]
Jul 22 19:38:59.354435 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.352261    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87a1b8b4-db7c-4e23-ac09-2afde18e19ed" containerName="main"
Jul 22 19:38:59.354435 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.352285    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="87a1b8b4-db7c-4e23-ac09-2afde18e19ed" containerName="main"
Jul 22 19:38:59.354435 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.352319    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38a324a-c4ad-4808-8f72-aeb01bb61596" containerName="main"
Jul 22 19:38:59.354435 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.352328    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38a324a-c4ad-4808-8f72-aeb01bb61596" containerName="main"
Jul 22 19:38:59.354435 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.352528    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38a324a-c4ad-4808-8f72-aeb01bb61596" containerName="main"
Jul 22 19:38:59.354435 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.352558    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="87a1b8b4-db7c-4e23-ac09-2afde18e19ed" containerName="main"
Jul 22 19:38:59.358366 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.358338    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.360898 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.360872    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-488695\"/\"openshift-service-ca.crt\""
Jul 22 19:38:59.361059 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.361045    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-488695\"/\"kube-root-ca.crt\""
Jul 22 19:38:59.361251 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.361233    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"]
Jul 22 19:38:59.361887 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.361871    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-488695\"/\"default-dockercfg-df2ff\""
Jul 22 19:38:59.361978 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.361960    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-488695\"/\"sub-test-model-488695-kserve-self-signed-certs\""
Jul 22 19:38:59.436426 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.436388    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tmp-dir\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.436426 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.436432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-home\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.436650 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.436452    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tls-certs\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.436650 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.436521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgk7s\" (UniqueName: \"kubernetes.io/projected/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-kube-api-access-xgk7s\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.436650 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.436603    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-model-cache\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.436775 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.436653    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-dshm\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.537966 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.537925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-home\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.537966 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.537963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tls-certs\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.538240 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.537993    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgk7s\" (UniqueName: \"kubernetes.io/projected/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-kube-api-access-xgk7s\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.538240 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.538052    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-model-cache\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.538240 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.538087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-dshm\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.538240 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.538147    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tmp-dir\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.538456 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.538391    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-home\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.538500 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.538473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-model-cache\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.538609 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.538592    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tmp-dir\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.540321 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.540292    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-dshm\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.540618 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.540601    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tls-certs\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.546470 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.546443    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgk7s\" (UniqueName: \"kubernetes.io/projected/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-kube-api-access-xgk7s\") pod \"sub-test-model-488695-kserve-6c876d8dcf-lj7fr\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") " pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.669055 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.668952    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:38:59.810116 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:38:59.810086    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"]
Jul 22 19:38:59.812439 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:38:59.812412    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca81d091_373c_4f18_b3b1_ce7b41d6deb8.slice/crio-fba947dd59df4424450844dba4e38ff06dba2f93dd1ad4a4cc403629a6787399 WatchSource:0}: Error finding container fba947dd59df4424450844dba4e38ff06dba2f93dd1ad4a4cc403629a6787399: Status 404 returned error can't find the container with id fba947dd59df4424450844dba4e38ff06dba2f93dd1ad4a4cc403629a6787399
Jul 22 19:39:00.003886 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:00.003793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr" event={"ID":"ca81d091-373c-4f18-b3b1-ce7b41d6deb8","Type":"ContainerStarted","Data":"3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb"}
Jul 22 19:39:00.003886 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:00.003839    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr" event={"ID":"ca81d091-373c-4f18-b3b1-ce7b41d6deb8","Type":"ContainerStarted","Data":"fba947dd59df4424450844dba4e38ff06dba2f93dd1ad4a4cc403629a6787399"}
Jul 22 19:39:00.022285 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:00.022232    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr" podStartSLOduration=1.022214567 podStartE2EDuration="1.022214567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:38:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:39:00.021560358 +0000 UTC m=+2890.053577889" watchObservedRunningTime="2026-07-22 19:39:00.022214567 +0000 UTC m=+2890.054232096"
Jul 22 19:39:03.242201 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:03.242163    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:39:09.669592 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:09.669557    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:39:09.669997 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:09.669605    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:39:09.682250 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:09.682228    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:39:10.065223 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:10.065146    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:39:15.140006 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.139973    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"]
Jul 22 19:39:15.143906 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.143881    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.147369 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.147346    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"default-dockercfg-4d4nd\""
Jul 22 19:39:15.147369 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.147364    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"kube-root-ca.crt\""
Jul 22 19:39:15.147574 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.147367    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"openshift-service-ca.crt\""
Jul 22 19:39:15.147574 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.147353    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"sub-test-model-f3fe27-kserve-self-signed-certs\""
Jul 22 19:39:15.154393 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.154368    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"]
Jul 22 19:39:15.172858 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.172827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-home\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.173013 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.172871    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba87d8b-b575-4034-838e-ce831ed133a3-tls-certs\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.173013 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.172931    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-tmp-dir\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.173013 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.172949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-dshm\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.173013 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.172993    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njwj9\" (UniqueName: \"kubernetes.io/projected/7ba87d8b-b575-4034-838e-ce831ed133a3-kube-api-access-njwj9\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.173158 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.173015    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-model-cache\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.273526 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.273487    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-tmp-dir\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.273526 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.273528    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-dshm\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.273785 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.273569    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njwj9\" (UniqueName: \"kubernetes.io/projected/7ba87d8b-b575-4034-838e-ce831ed133a3-kube-api-access-njwj9\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.273785 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.273591    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-model-cache\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.273785 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.273728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-home\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.273970 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.273828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba87d8b-b575-4034-838e-ce831ed133a3-tls-certs\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.274038 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.273963    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-tmp-dir\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.274038 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.274012    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-model-cache\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.274185 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.274166    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-home\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.275930 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.275914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-dshm\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.276219 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.276201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba87d8b-b575-4034-838e-ce831ed133a3-tls-certs\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.282009 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.281985    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njwj9\" (UniqueName: \"kubernetes.io/projected/7ba87d8b-b575-4034-838e-ce831ed133a3-kube-api-access-njwj9\") pod \"sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") " pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.455436 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.455343    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:15.594573 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:15.594542    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"]
Jul 22 19:39:15.596860 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:39:15.596827    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ba87d8b_b575_4034_838e_ce831ed133a3.slice/crio-4a17012e59741cd107a0504ec6d1b80ca362c8cfe2dfa593ecc82a94d1d0af6f WatchSource:0}: Error finding container 4a17012e59741cd107a0504ec6d1b80ca362c8cfe2dfa593ecc82a94d1d0af6f: Status 404 returned error can't find the container with id 4a17012e59741cd107a0504ec6d1b80ca362c8cfe2dfa593ecc82a94d1d0af6f
Jul 22 19:39:16.089780 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:16.089720    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn" event={"ID":"7ba87d8b-b575-4034-838e-ce831ed133a3","Type":"ContainerStarted","Data":"58e768ae9523649be7803bd972d4067084b1e5b4c75a839ed89ac71f9d82f880"}
Jul 22 19:39:16.089955 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:16.089790    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn" event={"ID":"7ba87d8b-b575-4034-838e-ce831ed133a3","Type":"ContainerStarted","Data":"4a17012e59741cd107a0504ec6d1b80ca362c8cfe2dfa593ecc82a94d1d0af6f"}
Jul 22 19:39:16.105929 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:16.105859    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn" podStartSLOduration=1.105839082 podStartE2EDuration="1.105839082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:39:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:39:16.105268199 +0000 UTC m=+2906.137285772" watchObservedRunningTime="2026-07-22 19:39:16.105839082 +0000 UTC m=+2906.137856616"
Jul 22 19:39:18.541802 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:18.541768    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:39:25.455663 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:25.455622    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:25.455663 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:25.455672    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:25.468357 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:25.468331    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:26.152922 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:26.152890    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:39:37.145489 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:37.145449    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:39:51.754262 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:51.754227    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:39:59.277503 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:39:59.277467    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"]
Jul 22 19:40:00.129404 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:00.129370    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"]
Jul 22 19:40:12.743290 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:12.743206    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:40:14.298257 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.298216    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr" podUID="ca81d091-373c-4f18-b3b1-ce7b41d6deb8" containerName="main" containerID="cri-o://3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb" gracePeriod=45
Jul 22 19:40:14.546639 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.546617    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:40:14.596455 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596159    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-home\") pod \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") "
Jul 22 19:40:14.596455 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596417    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tmp-dir\") pod \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") "
Jul 22 19:40:14.596455 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596449    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tls-certs\") pod \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") "
Jul 22 19:40:14.596699 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596519    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-model-cache\") pod \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") "
Jul 22 19:40:14.596699 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596574    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-dshm\") pod \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") "
Jul 22 19:40:14.596699 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596619    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgk7s\" (UniqueName: \"kubernetes.io/projected/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-kube-api-access-xgk7s\") pod \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\" (UID: \"ca81d091-373c-4f18-b3b1-ce7b41d6deb8\") "
Jul 22 19:40:14.596699 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596652    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca81d091-373c-4f18-b3b1-ce7b41d6deb8" (UID: "ca81d091-373c-4f18-b3b1-ce7b41d6deb8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:40:14.597007 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596880    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca81d091-373c-4f18-b3b1-ce7b41d6deb8" (UID: "ca81d091-373c-4f18-b3b1-ce7b41d6deb8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:40:14.597007 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596899    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:14.597007 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.596961    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-home" (OuterVolumeSpecName: "home") pod "ca81d091-373c-4f18-b3b1-ce7b41d6deb8" (UID: "ca81d091-373c-4f18-b3b1-ce7b41d6deb8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:40:14.598698 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.598668    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-dshm" (OuterVolumeSpecName: "dshm") pod "ca81d091-373c-4f18-b3b1-ce7b41d6deb8" (UID: "ca81d091-373c-4f18-b3b1-ce7b41d6deb8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:40:14.598867 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.598807    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-kube-api-access-xgk7s" (OuterVolumeSpecName: "kube-api-access-xgk7s") pod "ca81d091-373c-4f18-b3b1-ce7b41d6deb8" (UID: "ca81d091-373c-4f18-b3b1-ce7b41d6deb8"). InnerVolumeSpecName "kube-api-access-xgk7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:40:14.598867 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.598855    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca81d091-373c-4f18-b3b1-ce7b41d6deb8" (UID: "ca81d091-373c-4f18-b3b1-ce7b41d6deb8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:40:14.697628 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.697593    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:14.697628 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.697622    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:14.697628 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.697632    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgk7s\" (UniqueName: \"kubernetes.io/projected/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-kube-api-access-xgk7s\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:14.697879 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.697644    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:14.697879 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:14.697653    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca81d091-373c-4f18-b3b1-ce7b41d6deb8-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:15.150108 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.150053    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn" podUID="7ba87d8b-b575-4034-838e-ce831ed133a3" containerName="main" containerID="cri-o://58e768ae9523649be7803bd972d4067084b1e5b4c75a839ed89ac71f9d82f880" gracePeriod=45
Jul 22 19:40:15.389631 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.389579    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca81d091-373c-4f18-b3b1-ce7b41d6deb8" containerID="3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb" exitCode=0
Jul 22 19:40:15.390035 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.389659    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"
Jul 22 19:40:15.390035 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.389814    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr" event={"ID":"ca81d091-373c-4f18-b3b1-ce7b41d6deb8","Type":"ContainerDied","Data":"3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb"}
Jul 22 19:40:15.390035 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.389849    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr" event={"ID":"ca81d091-373c-4f18-b3b1-ce7b41d6deb8","Type":"ContainerDied","Data":"fba947dd59df4424450844dba4e38ff06dba2f93dd1ad4a4cc403629a6787399"}
Jul 22 19:40:15.390035 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.389870    2558 scope.go:117] "RemoveContainer" containerID="3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb"
Jul 22 19:40:15.391982 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.391953    2558 generic.go:356] "Generic (PLEG): container finished" podID="7ba87d8b-b575-4034-838e-ce831ed133a3" containerID="58e768ae9523649be7803bd972d4067084b1e5b4c75a839ed89ac71f9d82f880" exitCode=0
Jul 22 19:40:15.392141 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.392003    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn" event={"ID":"7ba87d8b-b575-4034-838e-ce831ed133a3","Type":"ContainerDied","Data":"58e768ae9523649be7803bd972d4067084b1e5b4c75a839ed89ac71f9d82f880"}
Jul 22 19:40:15.392141 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.392033    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn" event={"ID":"7ba87d8b-b575-4034-838e-ce831ed133a3","Type":"ContainerDied","Data":"4a17012e59741cd107a0504ec6d1b80ca362c8cfe2dfa593ecc82a94d1d0af6f"}
Jul 22 19:40:15.392141 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.392047    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a17012e59741cd107a0504ec6d1b80ca362c8cfe2dfa593ecc82a94d1d0af6f"
Jul 22 19:40:15.405742 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.405725    2558 scope.go:117] "RemoveContainer" containerID="3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb"
Jul 22 19:40:15.406032 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:40:15.406013    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb\": container with ID starting with 3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb not found: ID does not exist" containerID="3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb"
Jul 22 19:40:15.406092 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.406039    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb"} err="failed to get container status \"3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb\": rpc error: code = NotFound desc = could not find container \"3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb\": container with ID starting with 3df7fc669596e7039c41c195c504eae1122a5909b0e64450ff46c5e1731b85cb not found: ID does not exist"
Jul 22 19:40:15.408814 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.408798    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:40:15.418563 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.418543    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"]
Jul 22 19:40:15.421920 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.421902    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-488695/sub-test-model-488695-kserve-6c876d8dcf-lj7fr"]
Jul 22 19:40:15.504403 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504373    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-model-cache\") pod \"7ba87d8b-b575-4034-838e-ce831ed133a3\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") "
Jul 22 19:40:15.504592 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504417    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba87d8b-b575-4034-838e-ce831ed133a3-tls-certs\") pod \"7ba87d8b-b575-4034-838e-ce831ed133a3\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") "
Jul 22 19:40:15.504592 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504439    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-dshm\") pod \"7ba87d8b-b575-4034-838e-ce831ed133a3\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") "
Jul 22 19:40:15.504592 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504504    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-tmp-dir\") pod \"7ba87d8b-b575-4034-838e-ce831ed133a3\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") "
Jul 22 19:40:15.504592 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504545    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-home\") pod \"7ba87d8b-b575-4034-838e-ce831ed133a3\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") "
Jul 22 19:40:15.504592 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504568    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njwj9\" (UniqueName: \"kubernetes.io/projected/7ba87d8b-b575-4034-838e-ce831ed133a3-kube-api-access-njwj9\") pod \"7ba87d8b-b575-4034-838e-ce831ed133a3\" (UID: \"7ba87d8b-b575-4034-838e-ce831ed133a3\") "
Jul 22 19:40:15.504897 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504669    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-model-cache" (OuterVolumeSpecName: "model-cache") pod "7ba87d8b-b575-4034-838e-ce831ed133a3" (UID: "7ba87d8b-b575-4034-838e-ce831ed133a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:40:15.504897 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504732    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7ba87d8b-b575-4034-838e-ce831ed133a3" (UID: "7ba87d8b-b575-4034-838e-ce831ed133a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:40:15.504897 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504773    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-home" (OuterVolumeSpecName: "home") pod "7ba87d8b-b575-4034-838e-ce831ed133a3" (UID: "7ba87d8b-b575-4034-838e-ce831ed133a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:40:15.505076 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504945    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:15.505076 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504963    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:15.505076 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.504976    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:15.506560 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.506530    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-dshm" (OuterVolumeSpecName: "dshm") pod "7ba87d8b-b575-4034-838e-ce831ed133a3" (UID: "7ba87d8b-b575-4034-838e-ce831ed133a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:40:15.506659 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.506577    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba87d8b-b575-4034-838e-ce831ed133a3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7ba87d8b-b575-4034-838e-ce831ed133a3" (UID: "7ba87d8b-b575-4034-838e-ce831ed133a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:40:15.506659 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.506596    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ba87d8b-b575-4034-838e-ce831ed133a3-kube-api-access-njwj9" (OuterVolumeSpecName: "kube-api-access-njwj9") pod "7ba87d8b-b575-4034-838e-ce831ed133a3" (UID: "7ba87d8b-b575-4034-838e-ce831ed133a3"). InnerVolumeSpecName "kube-api-access-njwj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:40:15.605694 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.605661    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njwj9\" (UniqueName: \"kubernetes.io/projected/7ba87d8b-b575-4034-838e-ce831ed133a3-kube-api-access-njwj9\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:15.605694 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.605690    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ba87d8b-b575-4034-838e-ce831ed133a3-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:15.605694 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:15.605701    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ba87d8b-b575-4034-838e-ce831ed133a3-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:40:16.397859 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:16.397830    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"
Jul 22 19:40:16.418193 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:16.418167    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"]
Jul 22 19:40:16.421419 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:16.421396    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/sub-test-model-f3fe27-kserve-6c5d7c79b9-z4mcn"]
Jul 22 19:40:16.451559 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:16.451532    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ba87d8b-b575-4034-838e-ce831ed133a3" path="/var/lib/kubelet/pods/7ba87d8b-b575-4034-838e-ce831ed133a3/volumes"
Jul 22 19:40:16.452010 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:16.451993    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca81d091-373c-4f18-b3b1-ce7b41d6deb8" path="/var/lib/kubelet/pods/ca81d091-373c-4f18-b3b1-ce7b41d6deb8/volumes"
Jul 22 19:40:50.751206 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:50.751174    2558 scope.go:117] "RemoveContainer" containerID="d95321960d94d4c6246f270271c9837956ef288e6a5a7135d7ab238bdb907f26"
Jul 22 19:40:56.343931 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.343893    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"]
Jul 22 19:40:56.344468 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.344438    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca81d091-373c-4f18-b3b1-ce7b41d6deb8" containerName="main"
Jul 22 19:40:56.344468 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.344465    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca81d091-373c-4f18-b3b1-ce7b41d6deb8" containerName="main"
Jul 22 19:40:56.344662 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.344497    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba87d8b-b575-4034-838e-ce831ed133a3" containerName="main"
Jul 22 19:40:56.344662 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.344506    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba87d8b-b575-4034-838e-ce831ed133a3" containerName="main"
Jul 22 19:40:56.344662 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.344615    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba87d8b-b575-4034-838e-ce831ed133a3" containerName="main"
Jul 22 19:40:56.344662 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.344629    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca81d091-373c-4f18-b3b1-ce7b41d6deb8" containerName="main"
Jul 22 19:40:56.349228 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.349208    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.352576 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.352553    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"default-dockercfg-4d4nd\""
Jul 22 19:40:56.352677 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.352610    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"rate-test-model-f3fe27-kserve-self-signed-certs\""
Jul 22 19:40:56.352734 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.352688    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"kube-root-ca.crt\""
Jul 22 19:40:56.352734 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.352720    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f3fe27\"/\"openshift-service-ca.crt\""
Jul 22 19:40:56.357828 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.357802    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-home\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.357951 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.357841    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-dshm\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.357951 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.357861    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-model-cache\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.357951 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.357879    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tls-certs\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.358142 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.358018    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tmp-dir\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.358142 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.358045    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jlsm\" (UniqueName: \"kubernetes.io/projected/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-kube-api-access-6jlsm\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.358330 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.358309    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"]
Jul 22 19:40:56.459143 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.459112    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-home\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.459143 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.459143    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-dshm\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.459375 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.459162    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-model-cache\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.459375 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.459180    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tls-certs\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.459375 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.459256    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tmp-dir\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.459375 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.459284    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jlsm\" (UniqueName: \"kubernetes.io/projected/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-kube-api-access-6jlsm\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.459595 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.459515    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-home\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.459595 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.459567    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-model-cache\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.459692 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.459641    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tmp-dir\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.461486 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.461466    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-dshm\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.461626 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.461608    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tls-certs\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.466469 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.466450    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jlsm\" (UniqueName: \"kubernetes.io/projected/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-kube-api-access-6jlsm\") pod \"rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") " pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.659181 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.659152    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:40:56.796916 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:56.796890    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"]
Jul 22 19:40:56.799234 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:40:56.799205    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9632db32_309f_4fcf_9cb4_ba7ca98c38a0.slice/crio-5326f928d7964aa772e664d185ddc7bde2036b8aa7a7107511d1516c65aa4158 WatchSource:0}: Error finding container 5326f928d7964aa772e664d185ddc7bde2036b8aa7a7107511d1516c65aa4158: Status 404 returned error can't find the container with id 5326f928d7964aa772e664d185ddc7bde2036b8aa7a7107511d1516c65aa4158
Jul 22 19:40:57.608141 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:57.608106    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj" event={"ID":"9632db32-309f-4fcf-9cb4-ba7ca98c38a0","Type":"ContainerStarted","Data":"b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f"}
Jul 22 19:40:57.608141 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:57.608148    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj" event={"ID":"9632db32-309f-4fcf-9cb4-ba7ca98c38a0","Type":"ContainerStarted","Data":"5326f928d7964aa772e664d185ddc7bde2036b8aa7a7107511d1516c65aa4158"}
Jul 22 19:40:57.625191 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:40:57.625127    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj" podStartSLOduration=1.625108759 podStartE2EDuration="1.625108759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:40:57.62365733 +0000 UTC m=+3007.655674883" watchObservedRunningTime="2026-07-22 19:40:57.625108759 +0000 UTC m=+3007.657126290"
Jul 22 19:41:00.337929 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:00.337891    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:41:06.660306 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:06.660265    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:41:06.660306 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:06.660312    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:41:06.672983 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:06.672957    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:41:07.668563 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:07.668531    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:41:14.245296 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:14.245252    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:41:32.451570 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:32.451532    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:41:40.649696 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:40.649659    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"]
Jul 22 19:41:41.698063 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:41.698022    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7543-139835367290800"]
Jul 22 19:41:41.702183 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:41.702157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367290800"
Jul 22 19:41:41.705274 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:41.705246    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-t6m7l\""
Jul 22 19:41:41.706716 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:41.706688    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367290800"]
Jul 22 19:41:41.772472 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:41.772436    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp2wl\" (UniqueName: \"kubernetes.io/projected/24f6334d-3ed9-4217-91cf-63c971fa6c90-kube-api-access-kp2wl\") pod \"test-curl-7543-139835367290800\" (UID: \"24f6334d-3ed9-4217-91cf-63c971fa6c90\") " pod="opendatahub/test-curl-7543-139835367290800"
Jul 22 19:41:41.873229 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:41.873189    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2wl\" (UniqueName: \"kubernetes.io/projected/24f6334d-3ed9-4217-91cf-63c971fa6c90-kube-api-access-kp2wl\") pod \"test-curl-7543-139835367290800\" (UID: \"24f6334d-3ed9-4217-91cf-63c971fa6c90\") " pod="opendatahub/test-curl-7543-139835367290800"
Jul 22 19:41:41.880963 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:41.880939    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp2wl\" (UniqueName: \"kubernetes.io/projected/24f6334d-3ed9-4217-91cf-63c971fa6c90-kube-api-access-kp2wl\") pod \"test-curl-7543-139835367290800\" (UID: \"24f6334d-3ed9-4217-91cf-63c971fa6c90\") " pod="opendatahub/test-curl-7543-139835367290800"
Jul 22 19:41:42.014189 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:42.014091    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367290800"
Jul 22 19:41:42.151172 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:42.151146    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367290800"]
Jul 22 19:41:42.153772 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:41:42.153723    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24f6334d_3ed9_4217_91cf_63c971fa6c90.slice/crio-3ceca0c11598ea243fff140e76497322311514655f1cd91b543b329a8e6ebb00 WatchSource:0}: Error finding container 3ceca0c11598ea243fff140e76497322311514655f1cd91b543b329a8e6ebb00: Status 404 returned error can't find the container with id 3ceca0c11598ea243fff140e76497322311514655f1cd91b543b329a8e6ebb00
Jul 22 19:41:42.155099 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:42.155078    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 19:41:42.839997 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:42.839826    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367290800" event={"ID":"24f6334d-3ed9-4217-91cf-63c971fa6c90","Type":"ContainerStarted","Data":"3ceca0c11598ea243fff140e76497322311514655f1cd91b543b329a8e6ebb00"}
Jul 22 19:41:43.845948 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:43.845915    2558 generic.go:356] "Generic (PLEG): container finished" podID="24f6334d-3ed9-4217-91cf-63c971fa6c90" containerID="68e2b335a643c3394bc4f8b044372d5fea5518f3f286f504d1e4032802a1cea8" exitCode=0
Jul 22 19:41:43.846330 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:43.845972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367290800" event={"ID":"24f6334d-3ed9-4217-91cf-63c971fa6c90","Type":"ContainerDied","Data":"68e2b335a643c3394bc4f8b044372d5fea5518f3f286f504d1e4032802a1cea8"}
Jul 22 19:41:44.991526 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:44.991494    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367290800"
Jul 22 19:41:44.995943 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:44.995923    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp2wl\" (UniqueName: \"kubernetes.io/projected/24f6334d-3ed9-4217-91cf-63c971fa6c90-kube-api-access-kp2wl\") pod \"24f6334d-3ed9-4217-91cf-63c971fa6c90\" (UID: \"24f6334d-3ed9-4217-91cf-63c971fa6c90\") "
Jul 22 19:41:44.998029 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:44.998007    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24f6334d-3ed9-4217-91cf-63c971fa6c90-kube-api-access-kp2wl" (OuterVolumeSpecName: "kube-api-access-kp2wl") pod "24f6334d-3ed9-4217-91cf-63c971fa6c90" (UID: "24f6334d-3ed9-4217-91cf-63c971fa6c90"). InnerVolumeSpecName "kube-api-access-kp2wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:41:45.026224 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.026198    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7543-139835367290800_24f6334d-3ed9-4217-91cf-63c971fa6c90/test-curl-7543-139835367290800/0.log"
Jul 22 19:41:45.048148 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.048121    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7543-139835367290800"]
Jul 22 19:41:45.051666 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.051640    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7543-139835367290800"]
Jul 22 19:41:45.097032 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.096998    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp2wl\" (UniqueName: \"kubernetes.io/projected/24f6334d-3ed9-4217-91cf-63c971fa6c90-kube-api-access-kp2wl\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:45.169793 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.169706    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7543-139835367219760"]
Jul 22 19:41:45.170157 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.170142    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24f6334d-3ed9-4217-91cf-63c971fa6c90" containerName="test-curl-7543-139835367290800"
Jul 22 19:41:45.170200 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.170159    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="24f6334d-3ed9-4217-91cf-63c971fa6c90" containerName="test-curl-7543-139835367290800"
Jul 22 19:41:45.170241 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.170228    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="24f6334d-3ed9-4217-91cf-63c971fa6c90" containerName="test-curl-7543-139835367290800"
Jul 22 19:41:45.173234 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.173219    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367219760"
Jul 22 19:41:45.176962 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.176940    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367219760"]
Jul 22 19:41:45.197483 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.197449    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcr88\" (UniqueName: \"kubernetes.io/projected/9e242e0a-8aea-4c26-9960-b2652e0deb81-kube-api-access-qcr88\") pod \"test-curl-7543-139835367219760\" (UID: \"9e242e0a-8aea-4c26-9960-b2652e0deb81\") " pod="opendatahub/test-curl-7543-139835367219760"
Jul 22 19:41:45.298500 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.298463    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcr88\" (UniqueName: \"kubernetes.io/projected/9e242e0a-8aea-4c26-9960-b2652e0deb81-kube-api-access-qcr88\") pod \"test-curl-7543-139835367219760\" (UID: \"9e242e0a-8aea-4c26-9960-b2652e0deb81\") " pod="opendatahub/test-curl-7543-139835367219760"
Jul 22 19:41:45.306918 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.306882    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcr88\" (UniqueName: \"kubernetes.io/projected/9e242e0a-8aea-4c26-9960-b2652e0deb81-kube-api-access-qcr88\") pod \"test-curl-7543-139835367219760\" (UID: \"9e242e0a-8aea-4c26-9960-b2652e0deb81\") " pod="opendatahub/test-curl-7543-139835367219760"
Jul 22 19:41:45.484389 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.484293    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367219760"
Jul 22 19:41:45.618161 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.618137    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367219760"]
Jul 22 19:41:45.620092 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:41:45.620070    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e242e0a_8aea_4c26_9960_b2652e0deb81.slice/crio-9c4a7602133eda92e95d32174e667b6140d7574ea637461d8cdfb1e0eb318772 WatchSource:0}: Error finding container 9c4a7602133eda92e95d32174e667b6140d7574ea637461d8cdfb1e0eb318772: Status 404 returned error can't find the container with id 9c4a7602133eda92e95d32174e667b6140d7574ea637461d8cdfb1e0eb318772
Jul 22 19:41:45.857896 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.857868    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367290800"
Jul 22 19:41:45.858095 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.857871    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ceca0c11598ea243fff140e76497322311514655f1cd91b543b329a8e6ebb00"
Jul 22 19:41:45.859018 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:45.858984    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367219760" event={"ID":"9e242e0a-8aea-4c26-9960-b2652e0deb81","Type":"ContainerStarted","Data":"9c4a7602133eda92e95d32174e667b6140d7574ea637461d8cdfb1e0eb318772"}
Jul 22 19:41:45.868881 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:41:45.868857    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7543-139835367290800\" is forbidden: User \"system:node:ip-10-0-136-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-84.ec2.internal' and this object" podUID="24f6334d-3ed9-4217-91cf-63c971fa6c90" pod="opendatahub/test-curl-7543-139835367290800"
Jul 22 19:41:46.452021 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:46.451992    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24f6334d-3ed9-4217-91cf-63c971fa6c90" path="/var/lib/kubelet/pods/24f6334d-3ed9-4217-91cf-63c971fa6c90/volumes"
Jul 22 19:41:46.865229 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:46.865197    2558 generic.go:356] "Generic (PLEG): container finished" podID="9e242e0a-8aea-4c26-9960-b2652e0deb81" containerID="c0c05f1e7a1cb0d60e3e5a5b14c7733e18e6e7e4a6b6138ce24e3616d158bcf1" exitCode=0
Jul 22 19:41:46.865401 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:46.865287    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367219760" event={"ID":"9e242e0a-8aea-4c26-9960-b2652e0deb81","Type":"ContainerDied","Data":"c0c05f1e7a1cb0d60e3e5a5b14c7733e18e6e7e4a6b6138ce24e3616d158bcf1"}
Jul 22 19:41:47.245852 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:47.245767    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:41:48.007722 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.007699    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367219760"
Jul 22 19:41:48.021784 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.021735    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcr88\" (UniqueName: \"kubernetes.io/projected/9e242e0a-8aea-4c26-9960-b2652e0deb81-kube-api-access-qcr88\") pod \"9e242e0a-8aea-4c26-9960-b2652e0deb81\" (UID: \"9e242e0a-8aea-4c26-9960-b2652e0deb81\") "
Jul 22 19:41:48.024317 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.024290    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e242e0a-8aea-4c26-9960-b2652e0deb81-kube-api-access-qcr88" (OuterVolumeSpecName: "kube-api-access-qcr88") pod "9e242e0a-8aea-4c26-9960-b2652e0deb81" (UID: "9e242e0a-8aea-4c26-9960-b2652e0deb81"). InnerVolumeSpecName "kube-api-access-qcr88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:41:48.025259 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.025234    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7543-139835367219760_9e242e0a-8aea-4c26-9960-b2652e0deb81/test-curl-7543-139835367219760/0.log"
Jul 22 19:41:48.047362 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.047333    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7543-139835367219760"]
Jul 22 19:41:48.050083 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.050063    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7543-139835367219760"]
Jul 22 19:41:48.123513 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.123421    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcr88\" (UniqueName: \"kubernetes.io/projected/9e242e0a-8aea-4c26-9960-b2652e0deb81-kube-api-access-qcr88\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:48.206216 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.206168    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:48.206621 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.206605    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e242e0a-8aea-4c26-9960-b2652e0deb81" containerName="test-curl-7543-139835367219760"
Jul 22 19:41:48.206665 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.206623    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e242e0a-8aea-4c26-9960-b2652e0deb81" containerName="test-curl-7543-139835367219760"
Jul 22 19:41:48.206697 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.206682    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e242e0a-8aea-4c26-9960-b2652e0deb81" containerName="test-curl-7543-139835367219760"
Jul 22 19:41:48.209799 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.209776    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:48.215256 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.215230    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:48.224290 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.224268    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4svx\" (UniqueName: \"kubernetes.io/projected/0babece2-40be-4398-8b6f-6c46b584cf3a-kube-api-access-n4svx\") pod \"test-curl-7543-139835367310080\" (UID: \"0babece2-40be-4398-8b6f-6c46b584cf3a\") " pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:48.325048 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.324977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4svx\" (UniqueName: \"kubernetes.io/projected/0babece2-40be-4398-8b6f-6c46b584cf3a-kube-api-access-n4svx\") pod \"test-curl-7543-139835367310080\" (UID: \"0babece2-40be-4398-8b6f-6c46b584cf3a\") " pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:48.332284 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.332264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4svx\" (UniqueName: \"kubernetes.io/projected/0babece2-40be-4398-8b6f-6c46b584cf3a-kube-api-access-n4svx\") pod \"test-curl-7543-139835367310080\" (UID: \"0babece2-40be-4398-8b6f-6c46b584cf3a\") " pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:48.451762 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.451660    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e242e0a-8aea-4c26-9960-b2652e0deb81" path="/var/lib/kubelet/pods/9e242e0a-8aea-4c26-9960-b2652e0deb81/volumes"
Jul 22 19:41:48.521815 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.521784    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:48.658423 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.658394    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:48.660795 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:41:48.660767    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0babece2_40be_4398_8b6f_6c46b584cf3a.slice/crio-cb39c913851a134367ef0bbbb6f8ceeae94a5f3cf137a9c002deeefeefa54044 WatchSource:0}: Error finding container cb39c913851a134367ef0bbbb6f8ceeae94a5f3cf137a9c002deeefeefa54044: Status 404 returned error can't find the container with id cb39c913851a134367ef0bbbb6f8ceeae94a5f3cf137a9c002deeefeefa54044
Jul 22 19:41:48.876580 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.876543    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367310080" event={"ID":"0babece2-40be-4398-8b6f-6c46b584cf3a","Type":"ContainerStarted","Data":"cb39c913851a134367ef0bbbb6f8ceeae94a5f3cf137a9c002deeefeefa54044"}
Jul 22 19:41:48.877811 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.877789    2558 scope.go:117] "RemoveContainer" containerID="c0c05f1e7a1cb0d60e3e5a5b14c7733e18e6e7e4a6b6138ce24e3616d158bcf1"
Jul 22 19:41:48.877923 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:48.877798    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367219760"
Jul 22 19:41:49.884820 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:49.884780    2558 generic.go:356] "Generic (PLEG): container finished" podID="0babece2-40be-4398-8b6f-6c46b584cf3a" containerID="eee0cdba5fab83802b92e65294aec50d4cb9229cbe3f0df3861cfd737725a7ad" exitCode=0
Jul 22 19:41:49.885297 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:49.884869    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367310080" event={"ID":"0babece2-40be-4398-8b6f-6c46b584cf3a","Type":"ContainerDied","Data":"eee0cdba5fab83802b92e65294aec50d4cb9229cbe3f0df3861cfd737725a7ad"}
Jul 22 19:41:51.037483 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.037452    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:51.056243 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.056108    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7543-139835367310080_0babece2-40be-4398-8b6f-6c46b584cf3a/test-curl-7543-139835367310080/0.log"
Jul 22 19:41:51.078062 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.077989    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:51.080186 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.080157    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:51.149772 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.149710    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4svx\" (UniqueName: \"kubernetes.io/projected/0babece2-40be-4398-8b6f-6c46b584cf3a-kube-api-access-n4svx\") pod \"0babece2-40be-4398-8b6f-6c46b584cf3a\" (UID: \"0babece2-40be-4398-8b6f-6c46b584cf3a\") "
Jul 22 19:41:51.151806 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.151743    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0babece2-40be-4398-8b6f-6c46b584cf3a-kube-api-access-n4svx" (OuterVolumeSpecName: "kube-api-access-n4svx") pod "0babece2-40be-4398-8b6f-6c46b584cf3a" (UID: "0babece2-40be-4398-8b6f-6c46b584cf3a"). InnerVolumeSpecName "kube-api-access-n4svx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:41:51.197958 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.197922    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:51.198350 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.198337    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0babece2-40be-4398-8b6f-6c46b584cf3a" containerName="test-curl-7543-139835367310080"
Jul 22 19:41:51.198391 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.198351    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0babece2-40be-4398-8b6f-6c46b584cf3a" containerName="test-curl-7543-139835367310080"
Jul 22 19:41:51.198437 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.198427    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0babece2-40be-4398-8b6f-6c46b584cf3a" containerName="test-curl-7543-139835367310080"
Jul 22 19:41:51.201493 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.201470    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:51.205452 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.205422    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:51.250466 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.250426    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glbxt\" (UniqueName: \"kubernetes.io/projected/4238c8ca-b1d4-47b0-aeab-9b8e6e106056-kube-api-access-glbxt\") pod \"test-curl-7543-139835367310080\" (UID: \"4238c8ca-b1d4-47b0-aeab-9b8e6e106056\") " pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:51.250647 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.250570    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4svx\" (UniqueName: \"kubernetes.io/projected/0babece2-40be-4398-8b6f-6c46b584cf3a-kube-api-access-n4svx\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:51.352008 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.351975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glbxt\" (UniqueName: \"kubernetes.io/projected/4238c8ca-b1d4-47b0-aeab-9b8e6e106056-kube-api-access-glbxt\") pod \"test-curl-7543-139835367310080\" (UID: \"4238c8ca-b1d4-47b0-aeab-9b8e6e106056\") " pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:51.360043 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.360013    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glbxt\" (UniqueName: \"kubernetes.io/projected/4238c8ca-b1d4-47b0-aeab-9b8e6e106056-kube-api-access-glbxt\") pod \"test-curl-7543-139835367310080\" (UID: \"4238c8ca-b1d4-47b0-aeab-9b8e6e106056\") " pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:51.512435 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.512395    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:51.645774 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.645726    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:51.648069 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:41:51.648043    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4238c8ca_b1d4_47b0_aeab_9b8e6e106056.slice/crio-06c6fcd246ed096fd74c25a1dfc2ad3a1b1d0b16fd3888aa6c2b35bb2ece4166 WatchSource:0}: Error finding container 06c6fcd246ed096fd74c25a1dfc2ad3a1b1d0b16fd3888aa6c2b35bb2ece4166: Status 404 returned error can't find the container with id 06c6fcd246ed096fd74c25a1dfc2ad3a1b1d0b16fd3888aa6c2b35bb2ece4166
Jul 22 19:41:51.898680 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.898583    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367310080" event={"ID":"4238c8ca-b1d4-47b0-aeab-9b8e6e106056","Type":"ContainerStarted","Data":"06c6fcd246ed096fd74c25a1dfc2ad3a1b1d0b16fd3888aa6c2b35bb2ece4166"}
Jul 22 19:41:51.899866 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.899847    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:51.899945 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.899853    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb39c913851a134367ef0bbbb6f8ceeae94a5f3cf137a9c002deeefeefa54044"
Jul 22 19:41:51.911309 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:51.911274    2558 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7543-139835367310080" oldPodUID="0babece2-40be-4398-8b6f-6c46b584cf3a" podUID="4238c8ca-b1d4-47b0-aeab-9b8e6e106056"
Jul 22 19:41:52.451819 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:52.451741    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0babece2-40be-4398-8b6f-6c46b584cf3a" path="/var/lib/kubelet/pods/0babece2-40be-4398-8b6f-6c46b584cf3a/volumes"
Jul 22 19:41:52.906944 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:52.906910    2558 generic.go:356] "Generic (PLEG): container finished" podID="4238c8ca-b1d4-47b0-aeab-9b8e6e106056" containerID="79d204a1306edc2c7a309afccfcb95cfb995de9736b4a994c2d951a8c19f9ec8" exitCode=0
Jul 22 19:41:52.907156 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:52.906989    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367310080" event={"ID":"4238c8ca-b1d4-47b0-aeab-9b8e6e106056","Type":"ContainerDied","Data":"79d204a1306edc2c7a309afccfcb95cfb995de9736b4a994c2d951a8c19f9ec8"}
Jul 22 19:41:54.053216 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.053187    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:54.072169 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.072144    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7543-139835367310080_4238c8ca-b1d4-47b0-aeab-9b8e6e106056/test-curl-7543-139835367310080/0.log"
Jul 22 19:41:54.076392 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.076368    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glbxt\" (UniqueName: \"kubernetes.io/projected/4238c8ca-b1d4-47b0-aeab-9b8e6e106056-kube-api-access-glbxt\") pod \"4238c8ca-b1d4-47b0-aeab-9b8e6e106056\" (UID: \"4238c8ca-b1d4-47b0-aeab-9b8e6e106056\") "
Jul 22 19:41:54.078411 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.078388    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4238c8ca-b1d4-47b0-aeab-9b8e6e106056-kube-api-access-glbxt" (OuterVolumeSpecName: "kube-api-access-glbxt") pod "4238c8ca-b1d4-47b0-aeab-9b8e6e106056" (UID: "4238c8ca-b1d4-47b0-aeab-9b8e6e106056"). InnerVolumeSpecName "kube-api-access-glbxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:41:54.092944 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.092916    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:54.094762 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.094728    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7543-139835367310080"]
Jul 22 19:41:54.177743 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.177650    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glbxt\" (UniqueName: \"kubernetes.io/projected/4238c8ca-b1d4-47b0-aeab-9b8e6e106056-kube-api-access-glbxt\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:54.208592 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.208552    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:41:54.209019 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.209003    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4238c8ca-b1d4-47b0-aeab-9b8e6e106056" containerName="test-curl-7543-139835367310080"
Jul 22 19:41:54.209088 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.209021    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4238c8ca-b1d4-47b0-aeab-9b8e6e106056" containerName="test-curl-7543-139835367310080"
Jul 22 19:41:54.209132 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.209094    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4238c8ca-b1d4-47b0-aeab-9b8e6e106056" containerName="test-curl-7543-139835367310080"
Jul 22 19:41:54.212170 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.212139    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:54.219117 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.219041    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:41:54.279036 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.278998    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cm7x\" (UniqueName: \"kubernetes.io/projected/dc68c1f7-a374-4355-95c9-3bafe3603be5-kube-api-access-7cm7x\") pod \"test-curl-7543-139835367311088\" (UID: \"dc68c1f7-a374-4355-95c9-3bafe3603be5\") " pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:54.380401 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.380364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cm7x\" (UniqueName: \"kubernetes.io/projected/dc68c1f7-a374-4355-95c9-3bafe3603be5-kube-api-access-7cm7x\") pod \"test-curl-7543-139835367311088\" (UID: \"dc68c1f7-a374-4355-95c9-3bafe3603be5\") " pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:54.388436 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.388401    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cm7x\" (UniqueName: \"kubernetes.io/projected/dc68c1f7-a374-4355-95c9-3bafe3603be5-kube-api-access-7cm7x\") pod \"test-curl-7543-139835367311088\" (UID: \"dc68c1f7-a374-4355-95c9-3bafe3603be5\") " pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:54.452797 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.452671    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4238c8ca-b1d4-47b0-aeab-9b8e6e106056" path="/var/lib/kubelet/pods/4238c8ca-b1d4-47b0-aeab-9b8e6e106056/volumes"
Jul 22 19:41:54.523271 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.523230    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:54.658144 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.658117    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:41:54.662540 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:41:54.662503    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc68c1f7_a374_4355_95c9_3bafe3603be5.slice/crio-50808795e7347e806ff29a42e3260d04160ff33a3dfbde7c7a4882f4a691b1e9 WatchSource:0}: Error finding container 50808795e7347e806ff29a42e3260d04160ff33a3dfbde7c7a4882f4a691b1e9: Status 404 returned error can't find the container with id 50808795e7347e806ff29a42e3260d04160ff33a3dfbde7c7a4882f4a691b1e9
Jul 22 19:41:54.918808 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.918774    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367310080"
Jul 22 19:41:54.918808 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.918804    2558 scope.go:117] "RemoveContainer" containerID="79d204a1306edc2c7a309afccfcb95cfb995de9736b4a994c2d951a8c19f9ec8"
Jul 22 19:41:54.920162 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:54.919926    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367311088" event={"ID":"dc68c1f7-a374-4355-95c9-3bafe3603be5","Type":"ContainerStarted","Data":"50808795e7347e806ff29a42e3260d04160ff33a3dfbde7c7a4882f4a691b1e9"}
Jul 22 19:41:55.670571 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.670525    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj" podUID="9632db32-309f-4fcf-9cb4-ba7ca98c38a0" containerName="main" containerID="cri-o://b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f" gracePeriod=45
Jul 22 19:41:55.923289 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.923213    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:41:55.925578 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.925551    2558 generic.go:356] "Generic (PLEG): container finished" podID="9632db32-309f-4fcf-9cb4-ba7ca98c38a0" containerID="b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f" exitCode=0
Jul 22 19:41:55.925690 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.925587    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"
Jul 22 19:41:55.925690 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.925630    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj" event={"ID":"9632db32-309f-4fcf-9cb4-ba7ca98c38a0","Type":"ContainerDied","Data":"b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f"}
Jul 22 19:41:55.925690 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.925676    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj" event={"ID":"9632db32-309f-4fcf-9cb4-ba7ca98c38a0","Type":"ContainerDied","Data":"5326f928d7964aa772e664d185ddc7bde2036b8aa7a7107511d1516c65aa4158"}
Jul 22 19:41:55.925891 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.925695    2558 scope.go:117] "RemoveContainer" containerID="b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f"
Jul 22 19:41:55.928177 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.928153    2558 generic.go:356] "Generic (PLEG): container finished" podID="dc68c1f7-a374-4355-95c9-3bafe3603be5" containerID="6f68563d356d471a47d7426f79f069aa38ac8fab243ab2732840181d9b16346f" exitCode=0
Jul 22 19:41:55.928281 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.928200    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367311088" event={"ID":"dc68c1f7-a374-4355-95c9-3bafe3603be5","Type":"ContainerDied","Data":"6f68563d356d471a47d7426f79f069aa38ac8fab243ab2732840181d9b16346f"}
Jul 22 19:41:55.934131 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.934114    2558 scope.go:117] "RemoveContainer" containerID="b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f"
Jul 22 19:41:55.934384 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:41:55.934364    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f\": container with ID starting with b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f not found: ID does not exist" containerID="b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f"
Jul 22 19:41:55.934479 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.934392    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f"} err="failed to get container status \"b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f\": rpc error: code = NotFound desc = could not find container \"b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f\": container with ID starting with b3379294d140d9ede722c950aee3b265f82623b36afcc3a569935499de713b8f not found: ID does not exist"
Jul 22 19:41:55.993913 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.993882    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-home\") pod \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") "
Jul 22 19:41:55.994116 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.993934    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tmp-dir\") pod \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") "
Jul 22 19:41:55.994116 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.993975    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-dshm\") pod \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") "
Jul 22 19:41:55.994116 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.994020    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tls-certs\") pod \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") "
Jul 22 19:41:55.994116 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.994042    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-model-cache\") pod \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") "
Jul 22 19:41:55.994116 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.994074    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jlsm\" (UniqueName: \"kubernetes.io/projected/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-kube-api-access-6jlsm\") pod \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\" (UID: \"9632db32-309f-4fcf-9cb4-ba7ca98c38a0\") "
Jul 22 19:41:55.994389 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.994173    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-home" (OuterVolumeSpecName: "home") pod "9632db32-309f-4fcf-9cb4-ba7ca98c38a0" (UID: "9632db32-309f-4fcf-9cb4-ba7ca98c38a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:41:55.994389 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.994257    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9632db32-309f-4fcf-9cb4-ba7ca98c38a0" (UID: "9632db32-309f-4fcf-9cb4-ba7ca98c38a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:41:55.994499 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.994427    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:55.994499 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.994445    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:55.994609 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.994505    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "9632db32-309f-4fcf-9cb4-ba7ca98c38a0" (UID: "9632db32-309f-4fcf-9cb4-ba7ca98c38a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:41:55.996134 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.996107    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-dshm" (OuterVolumeSpecName: "dshm") pod "9632db32-309f-4fcf-9cb4-ba7ca98c38a0" (UID: "9632db32-309f-4fcf-9cb4-ba7ca98c38a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 19:41:55.996236 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.996175    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-kube-api-access-6jlsm" (OuterVolumeSpecName: "kube-api-access-6jlsm") pod "9632db32-309f-4fcf-9cb4-ba7ca98c38a0" (UID: "9632db32-309f-4fcf-9cb4-ba7ca98c38a0"). InnerVolumeSpecName "kube-api-access-6jlsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:41:55.996236 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:55.996213    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9632db32-309f-4fcf-9cb4-ba7ca98c38a0" (UID: "9632db32-309f-4fcf-9cb4-ba7ca98c38a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:41:56.095119 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:56.095081    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:56.095119 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:56.095112    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:56.095119 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:56.095123    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:56.095336 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:56.095133    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jlsm\" (UniqueName: \"kubernetes.io/projected/9632db32-309f-4fcf-9cb4-ba7ca98c38a0-kube-api-access-6jlsm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:56.247169 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:56.247136    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"]
Jul 22 19:41:56.250637 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:56.250615    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f3fe27/rate-test-model-f3fe27-kserve-999fcc8bc-lt5wj"]
Jul 22 19:41:56.452333 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:56.452300    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9632db32-309f-4fcf-9cb4-ba7ca98c38a0" path="/var/lib/kubelet/pods/9632db32-309f-4fcf-9cb4-ba7ca98c38a0/volumes"
Jul 22 19:41:57.095232 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.095206    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:57.118142 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.117845    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7543-139835367311088_dc68c1f7-a374-4355-95c9-3bafe3603be5/test-curl-7543-139835367311088/0.log"
Jul 22 19:41:57.139578 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.139550    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:41:57.144051 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.144025    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:41:57.204858 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.204778    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cm7x\" (UniqueName: \"kubernetes.io/projected/dc68c1f7-a374-4355-95c9-3bafe3603be5-kube-api-access-7cm7x\") pod \"dc68c1f7-a374-4355-95c9-3bafe3603be5\" (UID: \"dc68c1f7-a374-4355-95c9-3bafe3603be5\") "
Jul 22 19:41:57.206905 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.206875    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc68c1f7-a374-4355-95c9-3bafe3603be5-kube-api-access-7cm7x" (OuterVolumeSpecName: "kube-api-access-7cm7x") pod "dc68c1f7-a374-4355-95c9-3bafe3603be5" (UID: "dc68c1f7-a374-4355-95c9-3bafe3603be5"). InnerVolumeSpecName "kube-api-access-7cm7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:41:57.266383 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.266348    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:41:57.266815 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.266801    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9632db32-309f-4fcf-9cb4-ba7ca98c38a0" containerName="main"
Jul 22 19:41:57.266858 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.266817    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9632db32-309f-4fcf-9cb4-ba7ca98c38a0" containerName="main"
Jul 22 19:41:57.266858 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.266827    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc68c1f7-a374-4355-95c9-3bafe3603be5" containerName="test-curl-7543-139835367311088"
Jul 22 19:41:57.266858 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.266837    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc68c1f7-a374-4355-95c9-3bafe3603be5" containerName="test-curl-7543-139835367311088"
Jul 22 19:41:57.266965 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.266923    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9632db32-309f-4fcf-9cb4-ba7ca98c38a0" containerName="main"
Jul 22 19:41:57.266965 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.266934    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc68c1f7-a374-4355-95c9-3bafe3603be5" containerName="test-curl-7543-139835367311088"
Jul 22 19:41:57.269926 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.269909    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:57.275171 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.275137    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:41:57.305787 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.305733    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-942kz\" (UniqueName: \"kubernetes.io/projected/1976f905-a987-4ceb-b467-9b2e0ef4b622-kube-api-access-942kz\") pod \"test-curl-7543-139835367311088\" (UID: \"1976f905-a987-4ceb-b467-9b2e0ef4b622\") " pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:57.305937 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.305808    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cm7x\" (UniqueName: \"kubernetes.io/projected/dc68c1f7-a374-4355-95c9-3bafe3603be5-kube-api-access-7cm7x\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:41:57.407120 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.407089    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-942kz\" (UniqueName: \"kubernetes.io/projected/1976f905-a987-4ceb-b467-9b2e0ef4b622-kube-api-access-942kz\") pod \"test-curl-7543-139835367311088\" (UID: \"1976f905-a987-4ceb-b467-9b2e0ef4b622\") " pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:57.414572 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.414546    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-942kz\" (UniqueName: \"kubernetes.io/projected/1976f905-a987-4ceb-b467-9b2e0ef4b622-kube-api-access-942kz\") pod \"test-curl-7543-139835367311088\" (UID: \"1976f905-a987-4ceb-b467-9b2e0ef4b622\") " pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:57.581041 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.580952    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:57.712332 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.712308    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:41:57.714009 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:41:57.713982    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1976f905_a987_4ceb_b467_9b2e0ef4b622.slice/crio-97d631cc854dc70d9c0aaa2a77653ab1bdf6b897ce26bb7894521d5557a36364 WatchSource:0}: Error finding container 97d631cc854dc70d9c0aaa2a77653ab1bdf6b897ce26bb7894521d5557a36364: Status 404 returned error can't find the container with id 97d631cc854dc70d9c0aaa2a77653ab1bdf6b897ce26bb7894521d5557a36364
Jul 22 19:41:57.940937 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.940905    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:41:57.941141 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.940914    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50808795e7347e806ff29a42e3260d04160ff33a3dfbde7c7a4882f4a691b1e9"
Jul 22 19:41:57.942141 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.942110    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367311088" event={"ID":"1976f905-a987-4ceb-b467-9b2e0ef4b622","Type":"ContainerStarted","Data":"97d631cc854dc70d9c0aaa2a77653ab1bdf6b897ce26bb7894521d5557a36364"}
Jul 22 19:41:57.952269 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:57.952245    2558 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7543-139835367311088" oldPodUID="dc68c1f7-a374-4355-95c9-3bafe3603be5" podUID="1976f905-a987-4ceb-b467-9b2e0ef4b622"
Jul 22 19:41:58.451653 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:58.451623    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc68c1f7-a374-4355-95c9-3bafe3603be5" path="/var/lib/kubelet/pods/dc68c1f7-a374-4355-95c9-3bafe3603be5/volumes"
Jul 22 19:41:58.948989 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:58.948952    2558 generic.go:356] "Generic (PLEG): container finished" podID="1976f905-a987-4ceb-b467-9b2e0ef4b622" containerID="96ffaddf666e8091483eb28c7b498a8c2101eff5771138c6771cd56e76abd16d" exitCode=0
Jul 22 19:41:58.949163 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:41:58.949041    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7543-139835367311088" event={"ID":"1976f905-a987-4ceb-b467-9b2e0ef4b622","Type":"ContainerDied","Data":"96ffaddf666e8091483eb28c7b498a8c2101eff5771138c6771cd56e76abd16d"}
Jul 22 19:42:00.090932 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.090906    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:42:00.109553 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.109511    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7543-139835367311088_1976f905-a987-4ceb-b467-9b2e0ef4b622/test-curl-7543-139835367311088/0.log"
Jul 22 19:42:00.132179 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.132142    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:42:00.133250 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.133223    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-942kz\" (UniqueName: \"kubernetes.io/projected/1976f905-a987-4ceb-b467-9b2e0ef4b622-kube-api-access-942kz\") pod \"1976f905-a987-4ceb-b467-9b2e0ef4b622\" (UID: \"1976f905-a987-4ceb-b467-9b2e0ef4b622\") "
Jul 22 19:42:00.135339 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.135310    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1976f905-a987-4ceb-b467-9b2e0ef4b622-kube-api-access-942kz" (OuterVolumeSpecName: "kube-api-access-942kz") pod "1976f905-a987-4ceb-b467-9b2e0ef4b622" (UID: "1976f905-a987-4ceb-b467-9b2e0ef4b622"). InnerVolumeSpecName "kube-api-access-942kz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:42:00.135624 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.135603    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7543-139835367311088"]
Jul 22 19:42:00.234159 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.234062    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-942kz\" (UniqueName: \"kubernetes.io/projected/1976f905-a987-4ceb-b467-9b2e0ef4b622-kube-api-access-942kz\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:42:00.451994 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.451959    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1976f905-a987-4ceb-b467-9b2e0ef4b622" path="/var/lib/kubelet/pods/1976f905-a987-4ceb-b467-9b2e0ef4b622/volumes"
Jul 22 19:42:00.960816 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.960782    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7543-139835367311088"
Jul 22 19:42:00.960816 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:00.960820    2558 scope.go:117] "RemoveContainer" containerID="96ffaddf666e8091483eb28c7b498a8c2101eff5771138c6771cd56e76abd16d"
Jul 22 19:42:10.699355 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:10.699318    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"]
Jul 22 19:42:10.699876 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:10.699566    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq" podUID="c50f1e73-6b64-49dd-aedc-cae9cd4671ed" containerName="maas-api" containerID="cri-o://ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4" gracePeriod=30
Jul 22 19:42:10.724801 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:10.724762    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq" podUID="c50f1e73-6b64-49dd-aedc-cae9cd4671ed" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.63:8443/health\": dial tcp 10.132.0.63:8443: connect: connection refused"
Jul 22 19:42:10.951722 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:10.951653    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:42:11.013298 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.013271    2558 generic.go:356] "Generic (PLEG): container finished" podID="c50f1e73-6b64-49dd-aedc-cae9cd4671ed" containerID="ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4" exitCode=0
Jul 22 19:42:11.013477 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.013321    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"
Jul 22 19:42:11.013477 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.013361    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq" event={"ID":"c50f1e73-6b64-49dd-aedc-cae9cd4671ed","Type":"ContainerDied","Data":"ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4"}
Jul 22 19:42:11.013477 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.013419    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq" event={"ID":"c50f1e73-6b64-49dd-aedc-cae9cd4671ed","Type":"ContainerDied","Data":"e7de5f8e03228b985da84e3c4d19c1951fe879c59ba5d1a3da5d07461a34ea46"}
Jul 22 19:42:11.013477 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.013449    2558 scope.go:117] "RemoveContainer" containerID="ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4"
Jul 22 19:42:11.022071 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.022049    2558 scope.go:117] "RemoveContainer" containerID="ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4"
Jul 22 19:42:11.022336 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:42:11.022315    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4\": container with ID starting with ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4 not found: ID does not exist" containerID="ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4"
Jul 22 19:42:11.022388 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.022347    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4"} err="failed to get container status \"ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4\": rpc error: code = NotFound desc = could not find container \"ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4\": container with ID starting with ffb5491bf7ec93283f85b252e199f0e6685dd7705d4d4cafaae44c622ee0ebc4 not found: ID does not exist"
Jul 22 19:42:11.032732 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.032708    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blg48\" (UniqueName: \"kubernetes.io/projected/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-kube-api-access-blg48\") pod \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") "
Jul 22 19:42:11.032847 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.032784    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-openshift-service-ca\") pod \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") "
Jul 22 19:42:11.032847 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.032811    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-maas-api-tls\") pod \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\" (UID: \"c50f1e73-6b64-49dd-aedc-cae9cd4671ed\") "
Jul 22 19:42:11.033169 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.033142    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c50f1e73-6b64-49dd-aedc-cae9cd4671ed" (UID: "c50f1e73-6b64-49dd-aedc-cae9cd4671ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:42:11.034898 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.034865    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-kube-api-access-blg48" (OuterVolumeSpecName: "kube-api-access-blg48") pod "c50f1e73-6b64-49dd-aedc-cae9cd4671ed" (UID: "c50f1e73-6b64-49dd-aedc-cae9cd4671ed"). InnerVolumeSpecName "kube-api-access-blg48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:42:11.034898 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.034871    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c50f1e73-6b64-49dd-aedc-cae9cd4671ed" (UID: "c50f1e73-6b64-49dd-aedc-cae9cd4671ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:42:11.133630 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.133601    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blg48\" (UniqueName: \"kubernetes.io/projected/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-kube-api-access-blg48\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:42:11.133630 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.133627    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:42:11.133630 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.133638    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c50f1e73-6b64-49dd-aedc-cae9cd4671ed-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:42:11.334233 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.334202    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"]
Jul 22 19:42:11.338218 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:11.338193    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-488695-6fd5cdf9b4-qqpvq"]
Jul 22 19:42:12.452701 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:12.452670    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c50f1e73-6b64-49dd-aedc-cae9cd4671ed" path="/var/lib/kubelet/pods/c50f1e73-6b64-49dd-aedc-cae9cd4671ed/volumes"
Jul 22 19:42:14.042352 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:14.042312    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:42:32.652817 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.652727    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"]
Jul 22 19:42:32.655812 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.653043    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc" podUID="b63e7611-93b2-495b-ade6-0c897848000d" containerName="maas-api" containerID="cri-o://de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2" gracePeriod=30
Jul 22 19:42:32.763527 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.763484    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc" podUID="b63e7611-93b2-495b-ade6-0c897848000d" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.64:8443/health\": dial tcp 10.132.0.64:8443: connect: connection refused"
Jul 22 19:42:32.916093 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.916031    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:42:32.936503 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.936473    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b63e7611-93b2-495b-ade6-0c897848000d-openshift-service-ca\") pod \"b63e7611-93b2-495b-ade6-0c897848000d\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") "
Jul 22 19:42:32.936673 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.936601    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjq7g\" (UniqueName: \"kubernetes.io/projected/b63e7611-93b2-495b-ade6-0c897848000d-kube-api-access-vjq7g\") pod \"b63e7611-93b2-495b-ade6-0c897848000d\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") "
Jul 22 19:42:32.936673 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.936667    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b63e7611-93b2-495b-ade6-0c897848000d-maas-api-tls\") pod \"b63e7611-93b2-495b-ade6-0c897848000d\" (UID: \"b63e7611-93b2-495b-ade6-0c897848000d\") "
Jul 22 19:42:32.936906 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.936884    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b63e7611-93b2-495b-ade6-0c897848000d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b63e7611-93b2-495b-ade6-0c897848000d" (UID: "b63e7611-93b2-495b-ade6-0c897848000d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 19:42:32.938583 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.938549    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b63e7611-93b2-495b-ade6-0c897848000d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b63e7611-93b2-495b-ade6-0c897848000d" (UID: "b63e7611-93b2-495b-ade6-0c897848000d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 19:42:32.938711 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:32.938605    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b63e7611-93b2-495b-ade6-0c897848000d-kube-api-access-vjq7g" (OuterVolumeSpecName: "kube-api-access-vjq7g") pod "b63e7611-93b2-495b-ade6-0c897848000d" (UID: "b63e7611-93b2-495b-ade6-0c897848000d"). InnerVolumeSpecName "kube-api-access-vjq7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 19:42:33.037561 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.037525    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b63e7611-93b2-495b-ade6-0c897848000d-maas-api-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:42:33.037561 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.037554    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b63e7611-93b2-495b-ade6-0c897848000d-openshift-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:42:33.037561 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.037566    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjq7g\" (UniqueName: \"kubernetes.io/projected/b63e7611-93b2-495b-ade6-0c897848000d-kube-api-access-vjq7g\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\""
Jul 22 19:42:33.132063 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.132025    2558 generic.go:356] "Generic (PLEG): container finished" podID="b63e7611-93b2-495b-ade6-0c897848000d" containerID="de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2" exitCode=0
Jul 22 19:42:33.132239 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.132071    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"
Jul 22 19:42:33.132239 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.132116    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc" event={"ID":"b63e7611-93b2-495b-ade6-0c897848000d","Type":"ContainerDied","Data":"de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2"}
Jul 22 19:42:33.132239 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.132185    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc" event={"ID":"b63e7611-93b2-495b-ade6-0c897848000d","Type":"ContainerDied","Data":"abba33a626726ea634bbffb1984104e2c74d49869f54bf7e4fc7917bdeb94985"}
Jul 22 19:42:33.132239 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.132215    2558 scope.go:117] "RemoveContainer" containerID="de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2"
Jul 22 19:42:33.142233 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.142214    2558 scope.go:117] "RemoveContainer" containerID="de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2"
Jul 22 19:42:33.142476 ip-10-0-136-84 kubenswrapper[2558]: E0722 19:42:33.142457    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2\": container with ID starting with de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2 not found: ID does not exist" containerID="de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2"
Jul 22 19:42:33.142524 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.142484    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2"} err="failed to get container status \"de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2\": rpc error: code = NotFound desc = could not find container \"de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2\": container with ID starting with de4f97fc4908dd0eb9fcf1b59cb2927814a7c957923a32df0e20887ade1845b2 not found: ID does not exist"
Jul 22 19:42:33.152948 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.152926    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"]
Jul 22 19:42:33.155038 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:33.155015    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-f3fe27-86644694bb-7bnwc"]
Jul 22 19:42:34.454463 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:34.454432    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b63e7611-93b2-495b-ade6-0c897848000d" path="/var/lib/kubelet/pods/b63e7611-93b2-495b-ade6-0c897848000d/volumes"
Jul 22 19:42:35.305949 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:35.305911    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tgsbj"]
Jul 22 19:42:49.948509 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:49.948472    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g2v88_229e9fbf-12a8-432f-92a4-09eb4102d9b7/manager/0.log"
Jul 22 19:42:50.070867 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:50.070837    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4ccgg_998d7344-e0ba-4d16-a4ba-2c100d4c89f0/manager/0.log"
Jul 22 19:42:50.331701 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:50.331620    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jxwpr_763af33e-9b1b-4379-a81e-359da48f65cc/server/0.log"
Jul 22 19:42:50.463217 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:50.463192    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-npgn5_39eb5aeb-0895-4ac6-8817-317da4b8a0d4/manager/0.log"
Jul 22 19:42:50.730589 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:50.730561    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64cd98d4d8-nhlkc_2a60d740-3c37-4a1f-bcdd-1c773394b164/copy-manifests/0.log"
Jul 22 19:42:50.737034 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:50.736995    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64cd98d4d8-nhlkc_2a60d740-3c37-4a1f-bcdd-1c773394b164/manager/0.log"
Jul 22 19:42:51.357798 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:51.357713    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745795-gwjtw_883f0f0d-c089-4811-abdf-8b06a33b2bc1/cleanup/0.log"
Jul 22 19:42:51.480710 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:51.480685    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745810-2k8nq_818b9371-2c61-4f2e-b733-ad6517ea528b/cleanup/0.log"
Jul 22 19:42:53.057330 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:53.057297    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2l66k_f572820f-fda2-47df-a6fa-a8f0755fd945/manager/0.log"
Jul 22 19:42:53.176368 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:53.176341    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qds99_6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7/manager/0.log"
Jul 22 19:42:53.292457 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:53.292430    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vxzbv_8e267eb7-de24-44b0-a76b-6ca12c2cc105/kuadrant-console-plugin/0.log"
Jul 22 19:42:53.428347 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:53.428318    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jjwqh_cecdec2f-ea8d-468a-a388-ab01658a99ee/registry-server/0.log"
Jul 22 19:42:53.552071 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:53.552041    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wjgkx_61ed6eb2-badd-4022-b535-904568301275/manager/0.log"
Jul 22 19:42:53.664246 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:53.664213    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tgsbj_55ee36fe-f446-4197-a85e-699cdf65fe05/limitador/0.log"
Jul 22 19:42:53.785844 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:53.785769    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-677ml_1f93d907-7b1a-45c3-aa08-9342e2b50153/manager/0.log"
Jul 22 19:42:54.484846 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:54.484814    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86546cfdc9-lwsnj_32b1a91d-debe-48a5-8ed7-33199b3f7c10/kube-auth-proxy/0.log"
Jul 22 19:42:55.304563 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:42:55.304536    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-94khf_d8307b70-8785-47d0-ae69-f7833da861bd/main/0.log"
Jul 22 19:43:01.621694 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.621658    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zws4d/must-gather-8xhvv"]
Jul 22 19:43:01.622160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.622040    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c50f1e73-6b64-49dd-aedc-cae9cd4671ed" containerName="maas-api"
Jul 22 19:43:01.622160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.622052    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c50f1e73-6b64-49dd-aedc-cae9cd4671ed" containerName="maas-api"
Jul 22 19:43:01.622160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.622060    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1976f905-a987-4ceb-b467-9b2e0ef4b622" containerName="test-curl-7543-139835367311088"
Jul 22 19:43:01.622160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.622067    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1976f905-a987-4ceb-b467-9b2e0ef4b622" containerName="test-curl-7543-139835367311088"
Jul 22 19:43:01.622160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.622089    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63e7611-93b2-495b-ade6-0c897848000d" containerName="maas-api"
Jul 22 19:43:01.622160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.622094    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63e7611-93b2-495b-ade6-0c897848000d" containerName="maas-api"
Jul 22 19:43:01.622160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.622148    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1976f905-a987-4ceb-b467-9b2e0ef4b622" containerName="test-curl-7543-139835367311088"
Jul 22 19:43:01.622160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.622157    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c50f1e73-6b64-49dd-aedc-cae9cd4671ed" containerName="maas-api"
Jul 22 19:43:01.622160 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.622164    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b63e7611-93b2-495b-ade6-0c897848000d" containerName="maas-api"
Jul 22 19:43:01.626874 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.626853    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zws4d/must-gather-8xhvv"
Jul 22 19:43:01.629509 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.629479    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zws4d\"/\"kube-root-ca.crt\""
Jul 22 19:43:01.629668 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.629562    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zws4d\"/\"default-dockercfg-z92fq\""
Jul 22 19:43:01.629863 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.629840    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zws4d\"/\"openshift-service-ca.crt\""
Jul 22 19:43:01.636005 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.635984    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zws4d/must-gather-8xhvv"]
Jul 22 19:43:01.790048 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.790011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb7r4\" (UniqueName: \"kubernetes.io/projected/c702eeb4-a495-426b-8b94-f816b62449e9-kube-api-access-rb7r4\") pod \"must-gather-8xhvv\" (UID: \"c702eeb4-a495-426b-8b94-f816b62449e9\") " pod="openshift-must-gather-zws4d/must-gather-8xhvv"
Jul 22 19:43:01.790220 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.790063    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c702eeb4-a495-426b-8b94-f816b62449e9-must-gather-output\") pod \"must-gather-8xhvv\" (UID: \"c702eeb4-a495-426b-8b94-f816b62449e9\") " pod="openshift-must-gather-zws4d/must-gather-8xhvv"
Jul 22 19:43:01.891107 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.890999    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb7r4\" (UniqueName: \"kubernetes.io/projected/c702eeb4-a495-426b-8b94-f816b62449e9-kube-api-access-rb7r4\") pod \"must-gather-8xhvv\" (UID: \"c702eeb4-a495-426b-8b94-f816b62449e9\") " pod="openshift-must-gather-zws4d/must-gather-8xhvv"
Jul 22 19:43:01.891107 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.891058    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c702eeb4-a495-426b-8b94-f816b62449e9-must-gather-output\") pod \"must-gather-8xhvv\" (UID: \"c702eeb4-a495-426b-8b94-f816b62449e9\") " pod="openshift-must-gather-zws4d/must-gather-8xhvv"
Jul 22 19:43:01.891475 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.891453    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c702eeb4-a495-426b-8b94-f816b62449e9-must-gather-output\") pod \"must-gather-8xhvv\" (UID: \"c702eeb4-a495-426b-8b94-f816b62449e9\") " pod="openshift-must-gather-zws4d/must-gather-8xhvv"
Jul 22 19:43:01.898926 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.898902    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb7r4\" (UniqueName: \"kubernetes.io/projected/c702eeb4-a495-426b-8b94-f816b62449e9-kube-api-access-rb7r4\") pod \"must-gather-8xhvv\" (UID: \"c702eeb4-a495-426b-8b94-f816b62449e9\") " pod="openshift-must-gather-zws4d/must-gather-8xhvv"
Jul 22 19:43:01.937037 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:01.937002    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zws4d/must-gather-8xhvv"
Jul 22 19:43:02.075129 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:02.075100    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zws4d/must-gather-8xhvv"]
Jul 22 19:43:02.077236 ip-10-0-136-84 kubenswrapper[2558]: W0722 19:43:02.077206    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc702eeb4_a495_426b_8b94_f816b62449e9.slice/crio-f4c8998f3997a819e09d5fa306da4eb9f924224b644826db608aab0618fd5ca6 WatchSource:0}: Error finding container f4c8998f3997a819e09d5fa306da4eb9f924224b644826db608aab0618fd5ca6: Status 404 returned error can't find the container with id f4c8998f3997a819e09d5fa306da4eb9f924224b644826db608aab0618fd5ca6
Jul 22 19:43:02.290023 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:02.289936    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zws4d/must-gather-8xhvv" event={"ID":"c702eeb4-a495-426b-8b94-f816b62449e9","Type":"ContainerStarted","Data":"f4c8998f3997a819e09d5fa306da4eb9f924224b644826db608aab0618fd5ca6"}
Jul 22 19:43:03.297347 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:03.297267    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zws4d/must-gather-8xhvv" event={"ID":"c702eeb4-a495-426b-8b94-f816b62449e9","Type":"ContainerStarted","Data":"5f494d2739d5c3e3615026ef939b1ac34696721e9d5d230f82a75ba68cf6fab1"}
Jul 22 19:43:03.297347 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:03.297318    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zws4d/must-gather-8xhvv" event={"ID":"c702eeb4-a495-426b-8b94-f816b62449e9","Type":"ContainerStarted","Data":"331af8f821c766456e72685ffaf571e589f7d48cc506cb9f700f4273f71a21ff"}
Jul 22 19:43:03.312804 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:03.312711    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zws4d/must-gather-8xhvv" podStartSLOduration=1.470431516 podStartE2EDuration="2.312688087s" totalImagesPullingTime="842.256571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:43:03.312180235 +0000 UTC m=+3133.344197766" watchObservedRunningTime="2026-07-22 19:43:03.312688087 +0000 UTC m=+3133.344705618"
Jul 22 19:43:04.490235 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:04.490198    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-78bmc_a3a1f9ec-d356-47cb-857b-ec77d9a5a7e0/global-pull-secret-syncer/0.log"
Jul 22 19:43:04.628994 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:04.628962    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fq67r_3bd62269-edb1-4c5c-9b70-3c7ea48c6d8b/konnectivity-agent/0.log"
Jul 22 19:43:04.725222 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:04.725188    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-84.ec2.internal_84cbfcd8e120f068e999cf03e85b5712/haproxy/0.log"
Jul 22 19:43:10.111172 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:10.111136    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2l66k_f572820f-fda2-47df-a6fa-a8f0755fd945/manager/0.log"
Jul 22 19:43:10.141128 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:10.141100    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qds99_6c2d250d-35c8-4a03-8f6a-4d1e4f30c4f7/manager/0.log"
Jul 22 19:43:10.177065 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:10.176457    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-vxzbv_8e267eb7-de24-44b0-a76b-6ca12c2cc105/kuadrant-console-plugin/0.log"
Jul 22 19:43:10.229001 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:10.228940    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jjwqh_cecdec2f-ea8d-468a-a388-ab01658a99ee/registry-server/0.log"
Jul 22 19:43:10.440871 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:10.440501    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-wjgkx_61ed6eb2-badd-4022-b535-904568301275/manager/0.log"
Jul 22 19:43:10.470773 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:10.469841    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tgsbj_55ee36fe-f446-4197-a85e-699cdf65fe05/limitador/0.log"
Jul 22 19:43:10.548318 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:10.548281    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-677ml_1f93d907-7b1a-45c3-aa08-9342e2b50153/manager/0.log"
Jul 22 19:43:12.137351 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.137222    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_64a7f412-dee4-45d8-b395-e5432c1c45d6/alertmanager/0.log"
Jul 22 19:43:12.163571 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.163542    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_64a7f412-dee4-45d8-b395-e5432c1c45d6/config-reloader/0.log"
Jul 22 19:43:12.187103 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.187075    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_64a7f412-dee4-45d8-b395-e5432c1c45d6/kube-rbac-proxy-web/0.log"
Jul 22 19:43:12.209117 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.209086    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_64a7f412-dee4-45d8-b395-e5432c1c45d6/kube-rbac-proxy/0.log"
Jul 22 19:43:12.231997 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.231946    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_64a7f412-dee4-45d8-b395-e5432c1c45d6/kube-rbac-proxy-metric/0.log"
Jul 22 19:43:12.254411 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.254384    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_64a7f412-dee4-45d8-b395-e5432c1c45d6/prom-label-proxy/0.log"
Jul 22 19:43:12.279594 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.279518    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_64a7f412-dee4-45d8-b395-e5432c1c45d6/init-config-reloader/0.log"
Jul 22 19:43:12.472571 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.472455    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-rchnt_8e27f963-59ee-49df-ad95-64ae9ff428bf/monitoring-plugin/0.log"
Jul 22 19:43:12.674134 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.674072    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z5hdr_6dd1d8be-7df1-42a6-bd54-328cc22ea5b5/node-exporter/0.log"
Jul 22 19:43:12.695476 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.695448    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z5hdr_6dd1d8be-7df1-42a6-bd54-328cc22ea5b5/kube-rbac-proxy/0.log"
Jul 22 19:43:12.721941 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:12.721910    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z5hdr_6dd1d8be-7df1-42a6-bd54-328cc22ea5b5/init-textfile/0.log"
Jul 22 19:43:13.007092 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:13.007055    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-lx2gf_d9512c06-e290-4738-91cc-93fe55610fd6/prometheus-operator/0.log"
Jul 22 19:43:13.025923 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:13.025886    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-lx2gf_d9512c06-e290-4738-91cc-93fe55610fd6/kube-rbac-proxy/0.log"
Jul 22 19:43:13.052873 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:13.052842    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-wwl77_421cdc54-b59d-47d0-a379-0eb8d0325610/prometheus-operator-admission-webhook/0.log"
Jul 22 19:43:13.084711 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:13.084680    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fb4898477-cmv8m_be00b088-e3c7-4361-a6c6-3bd93357f698/telemeter-client/0.log"
Jul 22 19:43:13.106333 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:13.106308    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fb4898477-cmv8m_be00b088-e3c7-4361-a6c6-3bd93357f698/reload/0.log"
Jul 22 19:43:13.128644 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:13.128607    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fb4898477-cmv8m_be00b088-e3c7-4361-a6c6-3bd93357f698/kube-rbac-proxy/0.log"
Jul 22 19:43:14.387767 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.387684    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"]
Jul 22 19:43:14.396402 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.396366    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.402772 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.401857    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"]
Jul 22 19:43:14.419469 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.419433    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-4qrhh_dfc22088-efc9-4e37-926b-a00989038020/networking-console-plugin/0.log"
Jul 22 19:43:14.432248 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.432203    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-podres\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.432425 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.432280    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-sys\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.432425 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.432331    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtzcf\" (UniqueName: \"kubernetes.io/projected/b052c49d-e051-4833-aba5-63a21f4c0edf-kube-api-access-mtzcf\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.432425 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.432388    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-proc\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.432425 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.432409    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-lib-modules\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.533247 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.533210    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-proc\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.533736 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.533688    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-lib-modules\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.533878 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.533846    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-podres\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.533943 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.533927    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-sys\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.534016 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.533999    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtzcf\" (UniqueName: \"kubernetes.io/projected/b052c49d-e051-4833-aba5-63a21f4c0edf-kube-api-access-mtzcf\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.534141 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.533530    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-proc\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.534248 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.534157    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-lib-modules\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.534412 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.534386    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-podres\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.534560 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.534521    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b052c49d-e051-4833-aba5-63a21f4c0edf-sys\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.543333 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.543289    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtzcf\" (UniqueName: \"kubernetes.io/projected/b052c49d-e051-4833-aba5-63a21f4c0edf-kube-api-access-mtzcf\") pod \"perf-node-gather-daemonset-525hq\" (UID: \"b052c49d-e051-4833-aba5-63a21f4c0edf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.716964 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.716875    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:14.913199 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:14.912960    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"]
Jul 22 19:43:15.403945 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:15.403903    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq" event={"ID":"b052c49d-e051-4833-aba5-63a21f4c0edf","Type":"ContainerStarted","Data":"44865b3812cfd37c6f815fce36aed637bdc8623a3497e715199a6ea73517ba25"}
Jul 22 19:43:15.404447 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:15.403973    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq" event={"ID":"b052c49d-e051-4833-aba5-63a21f4c0edf","Type":"ContainerStarted","Data":"133c6c56074ba3793bccaefe6b1dc88df7dbb14c23097a98f94a1dcf1caad996"}
Jul 22 19:43:15.404813 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:15.404787    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:15.421096 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:15.421044    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq" podStartSLOduration=1.421029193 podStartE2EDuration="1.421029193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 19:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:43:15.419298643 +0000 UTC m=+3145.451316173" watchObservedRunningTime="2026-07-22 19:43:15.421029193 +0000 UTC m=+3145.453046723"
Jul 22 19:43:17.462131 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:17.462100    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z2tqt_5ef7993f-e88f-4430-86be-7a8367cdc27c/dns/0.log"
Jul 22 19:43:17.483940 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:17.483909    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z2tqt_5ef7993f-e88f-4430-86be-7a8367cdc27c/kube-rbac-proxy/0.log"
Jul 22 19:43:17.532766 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:17.532723    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l6nb4_a6bbcded-2352-4a2f-ab3a-a6715fabcc1e/dns-node-resolver/0.log"
Jul 22 19:43:18.150393 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:18.150362    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-98jjr_2106254d-8297-442a-b586-3b8813e4ab8b/node-ca/0.log"
Jul 22 19:43:19.374351 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:19.374319    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86546cfdc9-lwsnj_32b1a91d-debe-48a5-8ed7-33199b3f7c10/kube-auth-proxy/0.log"
Jul 22 19:43:20.209848 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:20.209815    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6bpcc_956d7809-f242-41c4-a268-f692575d0b17/serve-healthcheck-canary/0.log"
Jul 22 19:43:20.750732 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:20.750701    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-6kg4x_2bdf4133-70f4-4a6f-b987-e375778ee50e/insights-operator/0.log"
Jul 22 19:43:20.751617 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:20.751592    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-6kg4x_2bdf4133-70f4-4a6f-b987-e375778ee50e/insights-operator/1.log"
Jul 22 19:43:20.846522 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:20.846496    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jzqv4_6683dc1e-7f60-4af9-aae1-dd0b2848b9ac/kube-rbac-proxy/0.log"
Jul 22 19:43:20.867880 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:20.867848    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jzqv4_6683dc1e-7f60-4af9-aae1-dd0b2848b9ac/exporter/0.log"
Jul 22 19:43:20.891085 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:20.891057    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jzqv4_6683dc1e-7f60-4af9-aae1-dd0b2848b9ac/extractor/0.log"
Jul 22 19:43:22.431758 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:22.431709    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-525hq"
Jul 22 19:43:23.247630 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:23.247601    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g2v88_229e9fbf-12a8-432f-92a4-09eb4102d9b7/manager/0.log"
Jul 22 19:43:23.534648 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:23.534557    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4ccgg_998d7344-e0ba-4d16-a4ba-2c100d4c89f0/manager/0.log"
Jul 22 19:43:23.876267 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:23.876239    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jxwpr_763af33e-9b1b-4379-a81e-359da48f65cc/server/0.log"
Jul 22 19:43:23.973955 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:23.973838    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-npgn5_39eb5aeb-0895-4ac6-8817-317da4b8a0d4/manager/0.log"
Jul 22 19:43:24.150295 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:24.150202    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64cd98d4d8-nhlkc_2a60d740-3c37-4a1f-bcdd-1c773394b164/manager/0.log"
Jul 22 19:43:24.173938 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:24.173849    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64cd98d4d8-nhlkc_2a60d740-3c37-4a1f-bcdd-1c773394b164/copy-manifests/0.log"
Jul 22 19:43:25.443620 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:25.443592    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86546cfdc9-lwsnj_32b1a91d-debe-48a5-8ed7-33199b3f7c10/kube-auth-proxy/0.log"
Jul 22 19:43:27.062431 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:27.062401    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86546cfdc9-lwsnj_32b1a91d-debe-48a5-8ed7-33199b3f7c10/kube-auth-proxy/0.log"
Jul 22 19:43:30.416092 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:30.416058    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-sttj6_9d9c768d-8962-4886-89f1-0411e082417a/migrator/0.log"
Jul 22 19:43:30.436494 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:30.436467    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-sttj6_9d9c768d-8962-4886-89f1-0411e082417a/graceful-termination/0.log"
Jul 22 19:43:32.121943 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:32.121913    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s66xt_f5ce3f6a-854b-4e1a-85b0-8a860dead860/kube-multus-additional-cni-plugins/0.log"
Jul 22 19:43:32.146637 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:32.146609    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s66xt_f5ce3f6a-854b-4e1a-85b0-8a860dead860/egress-router-binary-copy/0.log"
Jul 22 19:43:32.172814 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:32.172787    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s66xt_f5ce3f6a-854b-4e1a-85b0-8a860dead860/cni-plugins/0.log"
Jul 22 19:43:32.194674 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:32.194648    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s66xt_f5ce3f6a-854b-4e1a-85b0-8a860dead860/bond-cni-plugin/0.log"
Jul 22 19:43:32.215150 ip-10-0-136-84 kubenswrapper[2558]: I0722 19:43:32.215123    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s66xt_f5ce3f6a-854b-4e1a-85b0-8a860dead860/routeoverride-cni/0.log"