Sep 10 15:39:58.637867 ip-10-0-136-61 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 15:39:58.638039 ip-10-0-136-61 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 15:39:58.638148 ip-10-0-136-61 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 15:39:58.638665 ip-10-0-136-61 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 15:40:08.740841 ip-10-0-136-61 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 15:40:08.740858 ip-10-0-136-61 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f2654f302d7b412aa77ef0c0d7f0ca69 --
Sep 10 15:41:43.963564 ip-10-0-136-61 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 15:41:44.378790 ip-10-0-136-61 kubenswrapper[2527]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 15:41:44.378790 ip-10-0-136-61 kubenswrapper[2527]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 15:41:44.378790 ip-10-0-136-61 kubenswrapper[2527]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 15:41:44.378790 ip-10-0-136-61 kubenswrapper[2527]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 15:41:44.378790 ip-10-0-136-61 kubenswrapper[2527]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 15:41:44.380293 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.380179    2527 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 15:41:44.387231 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387185    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387323    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387333    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387337    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387339    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387343    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387346    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387349    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387352    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387355    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387360    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387363    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387368    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387373    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387376    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387379    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387382    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387385    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387388    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:41:44.387400 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387391    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387393    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387396    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387399    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387403    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387405    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387409    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387417    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387420    2527 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387422    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387425    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387428    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387431    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387433    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387436    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387439    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387441    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387444    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387446    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387449    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:41:44.387871 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387452    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387454    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387457    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387459    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387463    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387470    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387473    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387476    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387479    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387482    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387485    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387488    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387492    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387498    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387503    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387506    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387509    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387512    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387515    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:41:44.388384 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387518    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387520    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387524    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387527    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387529    2527 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387532    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387535    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387538    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387541    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387543    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387546    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387551    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387553    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387556    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387559    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387561    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387564    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387567    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387569    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387572    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:41:44.388848 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387574    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387577    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387579    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387582    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387584    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387588    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387591    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387597    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387599    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387602    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387605    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387607    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387610    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387613    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387615    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387617    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387620    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387623    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387626    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387628    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:41:44.389349 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.387631    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388417    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388428    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388432    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388436    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388439    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388441    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388444    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388447    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388450    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388452    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388455    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388457    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388460    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388463    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388465    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388468    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388471    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388475    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:41:44.389837 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388478    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388483    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388486    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388489    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388491    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388494    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388497    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388499    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388502    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388504    2527 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388507    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388510    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388512    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388515    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388519    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388522    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388525    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388527    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388530    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388532    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:41:44.390366 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388535    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388538    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388540    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388543    2527 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388545    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388548    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388550    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388553    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388555    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388557    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388560    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388563    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388567    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388569    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388572    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388575    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388577    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388581    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388585    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:41:44.390863 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388588    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388591    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388593    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388596    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388598    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388601    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388603    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388606    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388609    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388612    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388614    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388617    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388620    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388622    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388625    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388627    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388630    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388632    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388634    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388637    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:41:44.391337 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388639    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388642    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388644    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388647    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388650    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388653    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388656    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388658    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388661    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388663    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388666    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388669    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388671    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388674    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388676    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388678    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388681    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388683    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388686    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:41:44.391820 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388688    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388691    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.388693    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389101    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389106    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389108    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389111    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389113    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389116    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389119    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389122    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389124    2527 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389127    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389144    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389147    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389150    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389153    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389156    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389158    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:41:44.392314 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389162    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389165    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389169    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389173    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389177    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389180    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389183    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389186    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389188    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389191    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389193    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389196    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389198    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389201    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389203    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389206    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389208    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389211    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389213    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389216    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:41:44.392785 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389219    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389221    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389226    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389229    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389231    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389234    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389237    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389239    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389242    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389244    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389247    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389250    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389252    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389255    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389258    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389261    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389263    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389266    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389268    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389271    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:41:44.393287 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389275    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389278    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389280    2527 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389283    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389285    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389288    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389291    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389293    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389295    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389298    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389302    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389305    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389308    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389311    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389315    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389318    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389320    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389324    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389327    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389329    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:41:44.393789 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389331    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389334    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389336    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389339    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389342    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389344    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389348    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389350    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389353    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389356    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389359    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389361    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389364    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389366    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389369    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389371    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389374    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389377    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389379    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389382    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:41:44.394296 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389384    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389386    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.389390    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389469    2527 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389517    2527 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389524    2527 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389531    2527 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389535    2527 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389541    2527 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389546    2527 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389551    2527 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389554    2527 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389557    2527 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389560    2527 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389564    2527 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389567    2527 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389570    2527 flags.go:64] FLAG: --cgroup-root=""
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389573    2527 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389576    2527 flags.go:64] FLAG: --client-ca-file=""
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389580    2527 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389583    2527 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389589    2527 flags.go:64] FLAG: --cluster-domain=""
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389591    2527 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 15:41:44.394787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389594    2527 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389597    2527 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389601    2527 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389605    2527 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389608    2527 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389612    2527 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389615    2527 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389618    2527 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389621    2527 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389624    2527 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389627    2527 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389630    2527 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389635    2527 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389638    2527 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389641    2527 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389645    2527 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389648    2527 flags.go:64] FLAG: --enable-server="true"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389652    2527 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389656    2527 flags.go:64] FLAG: --event-burst="100"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389659    2527 flags.go:64] FLAG: --event-qps="50"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389662    2527 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389665    2527 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389668    2527 flags.go:64] FLAG: --eviction-hard=""
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389672    2527 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 15:41:44.395400 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389675    2527 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389678    2527 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389681    2527 flags.go:64] FLAG: --eviction-soft=""
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389684    2527 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389687    2527 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389694    2527 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389697    2527 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389700    2527 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389703    2527 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389706    2527 flags.go:64] FLAG: --feature-gates=""
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389710    2527 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389713    2527 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389716    2527 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389719    2527 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389722    2527 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389724    2527 flags.go:64] FLAG: --help="false"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389727    2527 flags.go:64] FLAG: --hostname-override="ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389730    2527 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389733    2527 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389736    2527 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389739    2527 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389742    2527 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389745    2527 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 15:41:44.395988 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389749    2527 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389752    2527 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389756    2527 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389759    2527 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389762    2527 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389765    2527 flags.go:64] FLAG: --kube-reserved=""
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389768    2527 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389771    2527 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389774    2527 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389776    2527 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389779    2527 flags.go:64] FLAG: --lock-file=""
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389782    2527 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389785    2527 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389788    2527 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389794    2527 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389798    2527 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389801    2527 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389804    2527 flags.go:64] FLAG: --logging-format="text"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389806    2527 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389810    2527 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389812    2527 flags.go:64] FLAG: --manifest-url=""
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389815    2527 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389819    2527 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389822    2527 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389827    2527 flags.go:64] FLAG: --max-pods="110"
Sep 10 15:41:44.396609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389829    2527 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389832    2527 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389835    2527 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389838    2527 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389841    2527 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389845    2527 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389848    2527 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389859    2527 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389863    2527 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389867    2527 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389870    2527 flags.go:64] FLAG: --pod-cidr=""
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389873    2527 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389879    2527 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389881    2527 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389885    2527 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389887    2527 flags.go:64] FLAG: --port="10250"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389890    2527 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389893    2527 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0baa78e4144a18de6"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389897    2527 flags.go:64] FLAG: --qos-reserved=""
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389900    2527 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389903    2527 flags.go:64] FLAG: --register-node="true"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389906    2527 flags.go:64] FLAG: --register-with-taints=""
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389910    2527 flags.go:64] FLAG: --registry-burst="10"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389914    2527 flags.go:64] FLAG: --registry-qps="5"
Sep 10 15:41:44.397620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389917    2527 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389920    2527 flags.go:64] FLAG: --reserved-memory=""
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389924    2527 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389927    2527 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389930    2527 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389933    2527 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389936    2527 flags.go:64] FLAG: --runonce="false"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389939    2527 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389942    2527 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389945    2527 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389948    2527 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389951    2527 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389955    2527 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389958    2527 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389961    2527 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389965    2527 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389969    2527 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389971    2527 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389976    2527 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389979    2527 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389982    2527 flags.go:64] FLAG: --system-cgroups=""
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389985    2527 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389990    2527 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389993    2527 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.389996    2527 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 15:41:44.398687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390001    2527 flags.go:64] FLAG: --tls-min-version=""
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390003    2527 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390006    2527 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390009    2527 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390012    2527 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390016    2527 flags.go:64] FLAG: --v="2"
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390019    2527 flags.go:64] FLAG: --version="false"
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390024    2527 flags.go:64] FLAG: --vmodule=""
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390029    2527 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390032    2527 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390165    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390171    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390175    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390178    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390181    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390184    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390187    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390189    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390192    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390194    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390197    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390200    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:41:44.399561 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390204    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390206    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390209    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390214    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390217    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390219    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390222    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390225    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390227    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390230    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390233    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390236    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390240    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390244    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390248    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390252    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390254    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390258    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390261    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:41:44.400160 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390264    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390267    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390269    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390272    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390274    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390277    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390280    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390283    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390287    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390291    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390294    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390297    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390300    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390304    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390307    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390310    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390314    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390317    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390319    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390322    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:41:44.400936 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390325    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390328    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390331    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390333    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390336    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390338    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390341    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390344    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390346    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390349    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390352    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390356    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390359    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390362    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390364    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390367    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390370    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390372    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390375    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:41:44.401781 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390378    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390381    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390383    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390386    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390388    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390391    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390395    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390398    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390400    2527 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390404    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390407    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390409    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390412    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390416    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390420    2527 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390423    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390425    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390428    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390430    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390433    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:41:44.402451 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390435    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390438    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390441    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390443    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390446    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390448    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390451    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390454    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.390457    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.390462    2527 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.399198    2527 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.399223    2527 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.400301    2527 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.400319    2527 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400417    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:41:44.403277 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400427    2527 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400432    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400437    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400442    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400447    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400451    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400455    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400461    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400465    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400469    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400473    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400478    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400482    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400487    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400491    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400496    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400500    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400505    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400508    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400513    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:41:44.403914 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400518    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400522    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400526    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400533    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400540    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400545    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400550    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400555    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400561    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400566    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400570    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400575    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400579    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400584    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400588    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400592    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400596    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400602    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400609    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:41:44.404712 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400614    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400618    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400623    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400627    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400631    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400635    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400640    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400644    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400648    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400652    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400657    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400661    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400665    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400672    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400677    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400682    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400687    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400693    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400697    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:41:44.405237 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400701    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400706    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400711    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400715    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400719    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400723    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400727    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400731    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400735    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400739    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400744    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400748    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400752    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400756    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400761    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400765    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400768    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400772    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400776    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:41:44.405764 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400781    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400785    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400789    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400793    2527 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400797    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400802    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400807    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400811    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400815    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400819    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400824    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400828    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400832    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400837    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400842    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400846    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400850    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400855    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400859    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400863    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:41:44.406299 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.400867    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.400875    2527 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401154    2527 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401165    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401171    2527 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401176    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401180    2527 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401184    2527 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401188    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401192    2527 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401197    2527 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401201    2527 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401207    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401212    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 15:41:44.406853 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401219    2527 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401224    2527 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401230    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401234    2527 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401239    2527 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401244    2527 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401248    2527 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401253    2527 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401257    2527 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401261    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401266    2527 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401271    2527 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401275    2527 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401280    2527 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401285    2527 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401289    2527 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401293    2527 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401298    2527 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401302    2527 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401306    2527 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 15:41:44.407258 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401310    2527 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401314    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401318    2527 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401323    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401327    2527 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401331    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401335    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401339    2527 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401344    2527 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401348    2527 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401352    2527 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401356    2527 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401361    2527 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401365    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401369    2527 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401373    2527 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401377    2527 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401381    2527 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401387    2527 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401391    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 15:41:44.407778 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401395    2527 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401399    2527 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401403    2527 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401407    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401412    2527 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401417    2527 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401421    2527 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401425    2527 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401429    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401433    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401437    2527 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401441    2527 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401445    2527 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401449    2527 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401453    2527 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401458    2527 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401462    2527 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401466    2527 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401470    2527 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401474    2527 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 15:41:44.408612 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401478    2527 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401482    2527 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401486    2527 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401490    2527 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401494    2527 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401498    2527 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401502    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401506    2527 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401511    2527 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401515    2527 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401519    2527 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401523    2527 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401527    2527 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401531    2527 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401535    2527 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401539    2527 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401544    2527 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401549    2527 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401554    2527 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 15:41:44.409152 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401558    2527 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401562    2527 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401566    2527 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401570    2527 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401575    2527 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401579    2527 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401585    2527 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:44.401592    2527 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.401600    2527 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.401807    2527 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.404395    2527 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.405528    2527 server.go:1019] "Starting client certificate rotation"
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.405639    2527 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 15:41:44.409620 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.405684    2527 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 15:41:44.410126 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.408524    2527 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 15:41:44.411172 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.411116    2527 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 15:41:44.421018 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.420995    2527 log.go:25] "Validated CRI v1 runtime API"
Sep 10 15:41:44.425845 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.425823    2527 log.go:25] "Validated CRI v1 image API"
Sep 10 15:41:44.426740 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.426722    2527 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 15:41:44.431164 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.431120    2527 fs.go:135] Filesystem UUIDs: map[2e5f15f0-9193-4871-ac44-f94f903310e7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8f0668b5-5fd3-4caf-a060-5d79d9f2b7dd:/dev/nvme0n1p3]
Sep 10 15:41:44.431262 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.431161    2527 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 15:41:44.434426 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.434399    2527 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 15:41:44.437394 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.437256    2527 manager.go:216] Machine: {Timestamp:2026-09-10 15:41:44.435912676 +0000 UTC m=+0.352037566 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099902 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25006f4538e70c0a18935720fb4cda SystemUUID:ec25006f-4538-e70c-0a18-935720fb4cda BootID:f2654f30-2d7b-412a-a77e-f0c0d7f0ca69 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9c:ac:7a:8e:75 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9c:ac:7a:8e:75 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:72:c8:5b:6e:68 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 15:41:44.437394 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.437384    2527 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 15:41:44.437607 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.437590    2527 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 15:41:44.440532 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.440496    2527 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 15:41:44.440689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.440535    2527 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-61.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 15:41:44.441563 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.441550    2527 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 15:41:44.441609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.441566    2527 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 15:41:44.441609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.441574    2527 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 15:41:44.442053 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.442042    2527 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 15:41:44.443288 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.443274    2527 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 15:41:44.447283 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.447268    2527 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 15:41:44.447410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.447400    2527 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 15:41:44.447440 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.447433    2527 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 15:41:44.447468 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.447443    2527 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 15:41:44.447468 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.447454    2527 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 15:41:44.447468 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.447465    2527 kubelet.go:398] "Adding apiserver pod source"
Sep 10 15:41:44.447581 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.447474    2527 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 15:41:44.447948 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.447928    2527 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 15:41:44.448048 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.447956    2527 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 15:41:44.450296 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.450272    2527 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 15:41:44.452969 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.452954    2527 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 15:41:44.453029 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.452985    2527 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 15:41:44.453179 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453169    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 15:41:44.453213 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453185    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 15:41:44.453213 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453191    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 15:41:44.453213 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453197    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 15:41:44.453213 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453205    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 15:41:44.453213 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453210    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 15:41:44.453831 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453818    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 15:41:44.453831 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453828    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 15:41:44.453916 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453841    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 15:41:44.453916 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453848    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 15:41:44.453916 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453860    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 15:41:44.453916 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.453869    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 15:41:44.454772 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.454763    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 15:41:44.454820 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.454773    2527 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 15:41:44.458234 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.458207    2527 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-61.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 15:41:44.458621 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.458586    2527 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 15:41:44.458807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.458786    2527 server.go:1268] "Started kubelet"
Sep 10 15:41:44.458807 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.458787    2527 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-61.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 15:41:44.459435 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.459405    2527 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 15:41:44.459523 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.459471    2527 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 15:41:44.459930 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.459899    2527 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 15:41:44.460264 ip-10-0-136-61 systemd[1]: Started Kubernetes Kubelet.
Sep 10 15:41:44.460964 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.460945    2527 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 15:41:44.464196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.464177    2527 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 15:41:44.465514 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.465458    2527 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 15:41:44.465874 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.465856    2527 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 15:41:44.466071 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.466050    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:44.466782 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.466495    2527 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 15:41:44.466782 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.466672    2527 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 15:41:44.466782 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.466690    2527 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 15:41:44.466782 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.466776    2527 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 15:41:44.466782 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.466786    2527 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 15:41:44.467346 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.467328    2527 factory.go:55] Registering systemd factory
Sep 10 15:41:44.467346 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.467348    2527 factory.go:223] Registration of the systemd container factory successfully
Sep 10 15:41:44.467449 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.467331    2527 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 15:41:44.467960 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.467830    2527 factory.go:153] Registering CRI-O factory
Sep 10 15:41:44.467960 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.467848    2527 factory.go:223] Registration of the crio container factory successfully
Sep 10 15:41:44.467960 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.467936    2527 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 15:41:44.467960 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.467953    2527 factory.go:103] Registering Raw factory
Sep 10 15:41:44.467960 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.467963    2527 manager.go:1196] Started watching for new ooms in manager
Sep 10 15:41:44.468380 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.468364    2527 manager.go:349] Starting recovery of all containers
Sep 10 15:41:44.468700 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.468681    2527 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-61.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 15:41:44.468771 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.468733    2527 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 15:41:44.469934 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.468929    2527 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-61.ec2.internal.18d3fff4efef3abc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-61.ec2.internal,UID:ip-10-0-136-61.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-61.ec2.internal,},FirstTimestamp:2026-09-10 15:41:44.458754748 +0000 UTC m=+0.374879620,LastTimestamp:2026-09-10 15:41:44.458754748 +0000 UTC m=+0.374879620,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-61.ec2.internal,}"
Sep 10 15:41:44.479002 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.478984    2527 manager.go:354] Recovery completed
Sep 10 15:41:44.485350 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.485321    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:41:44.488048 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.488020    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:41:44.488160 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.488060    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:41:44.488160 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.488073    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:41:44.488160 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.488108    2527 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gtcxl"
Sep 10 15:41:44.488869 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.488852    2527 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 15:41:44.488869 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.488869    2527 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 15:41:44.488975 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.488888    2527 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 15:41:44.488975 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.488887    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:41:44.489800 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.489782    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:41:44.489899 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.489816    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:41:44.489899 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.489842    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:41:44.490277 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.490163    2527 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-61.ec2.internal.18d3fff4f1ae17fe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-61.ec2.internal,UID:ip-10-0-136-61.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-61.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-61.ec2.internal,},FirstTimestamp:2026-09-10 15:41:44.488040446 +0000 UTC m=+0.404165317,LastTimestamp:2026-09-10 15:41:44.488040446 +0000 UTC m=+0.404165317,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-61.ec2.internal,}"
Sep 10 15:41:44.491054 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.491036    2527 policy_none.go:49] "None policy: Start"
Sep 10 15:41:44.491054 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.491053    2527 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 15:41:44.491206 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.491064    2527 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 15:41:44.493233 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.493220    2527 policy_none.go:47] "Start"
Sep 10 15:41:44.494977 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.494956    2527 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gtcxl"
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.534810    2527 manager.go:337] "Starting Device Plugin manager"
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.534855    2527 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.534866    2527 server.go:87] "Starting device plugin registration server"
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.535229    2527 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.535243    2527 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.535403    2527 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.535489    2527 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.535499    2527 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.536090    2527 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 15:41:44.536461 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.536169    2527 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:44.555522 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.555482    2527 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 15:41:44.556776 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.556746    2527 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 15:41:44.556914 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.556816    2527 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 15:41:44.556914 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.556846    2527 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 15:41:44.556914 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.556895    2527 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 15:41:44.559245 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.559219    2527 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:41:44.635490 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.635374    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:41:44.636455 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.636433    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:41:44.636553 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.636476    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:41:44.636553 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.636491    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:41:44.636553 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.636513    2527 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.646244 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.646221    2527 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.646324 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.646252    2527 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-61.ec2.internal\": node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:44.657083 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.657061    2527 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal"]
Sep 10 15:41:44.657189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.657126    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:41:44.658523 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.658508    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:41:44.658597 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.658535    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:41:44.658597 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.658544    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:41:44.660317 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.660301    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:44.660955 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.660941    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:41:44.661076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.661062    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.661110 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.661098    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:41:44.661847 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.661827    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:41:44.661906 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.661862    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:41:44.661906 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.661830    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:41:44.661906 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.661900    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:41:44.662043 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.661915    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:41:44.662043 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.661876    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:41:44.664305 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.664286    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.664396 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.664318    2527 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 15:41:44.665099 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.665082    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 15:41:44.665197 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.665117    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 15:41:44.665197 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.665155    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeHasSufficientPID"
Sep 10 15:41:44.667331 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.667310    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/72a61b7f916a5c74ee03e2f7481d45ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal\" (UID: \"72a61b7f916a5c74ee03e2f7481d45ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.667452 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.667340    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72a61b7f916a5c74ee03e2f7481d45ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal\" (UID: \"72a61b7f916a5c74ee03e2f7481d45ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.667452 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.667378    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9c9407f7c27661a1f8778abcf84b0b26-config\") pod \"kube-apiserver-proxy-ip-10-0-136-61.ec2.internal\" (UID: \"9c9407f7c27661a1f8778abcf84b0b26\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.690257 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.690229    2527 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-61.ec2.internal\" not found" node="ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.695271 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.695253    2527 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-61.ec2.internal\" not found" node="ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.760810 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.760779    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:44.768096 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.768063    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/72a61b7f916a5c74ee03e2f7481d45ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal\" (UID: \"72a61b7f916a5c74ee03e2f7481d45ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.768257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.768103    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72a61b7f916a5c74ee03e2f7481d45ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal\" (UID: \"72a61b7f916a5c74ee03e2f7481d45ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.768257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.768128    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9c9407f7c27661a1f8778abcf84b0b26-config\") pod \"kube-apiserver-proxy-ip-10-0-136-61.ec2.internal\" (UID: \"9c9407f7c27661a1f8778abcf84b0b26\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.768257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.768202    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9c9407f7c27661a1f8778abcf84b0b26-config\") pod \"kube-apiserver-proxy-ip-10-0-136-61.ec2.internal\" (UID: \"9c9407f7c27661a1f8778abcf84b0b26\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.768257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.768221    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/72a61b7f916a5c74ee03e2f7481d45ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal\" (UID: \"72a61b7f916a5c74ee03e2f7481d45ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.768442 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.768201    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72a61b7f916a5c74ee03e2f7481d45ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal\" (UID: \"72a61b7f916a5c74ee03e2f7481d45ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.861315 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.861259    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:44.961586 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:44.961560    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:44.993199 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.993130    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:44.998902 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:44.998460    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:45.061952 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.061893    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:45.169700 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.169663    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:45.270468 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.270356    2527 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-61.ec2.internal\" not found"
Sep 10 15:41:45.322093 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.322055    2527 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:41:45.368241 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.368213    2527 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:45.382483 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.382455    2527 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 15:41:45.384399 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.384375    2527 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal"
Sep 10 15:41:45.393662 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.393634    2527 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 15:41:45.405089 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.405049    2527 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 15:41:45.405296 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.405272    2527 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 15:41:45.405342 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.405272    2527 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 15:41:45.443973 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.443945    2527 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:41:45.447737 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.447711    2527 apiserver.go:52] "Watching apiserver"
Sep 10 15:41:45.465765 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.465725    2527 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 15:41:45.467352 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.467329    2527 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 15:41:45.467829 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.467803    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-v96j6","openshift-cluster-node-tuning-operator/tuned-s2xpk","openshift-image-registry/node-ca-xfqhc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal","openshift-multus/multus-additional-cni-plugins-ps5pl","openshift-network-operator/iptables-alerter-h9h7f","openshift-ovn-kubernetes/ovnkube-node-zb2zb","kube-system/global-pull-secret-syncer-z9ct5","kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65","openshift-dns/node-resolver-4djmx","openshift-multus/multus-sv2hv","openshift-multus/network-metrics-daemon-5c6r7","openshift-network-diagnostics/network-check-target-qswkd"]
Sep 10 15:41:45.470487 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.470471    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:41:45.472852 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.472828    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 15:41:45.472999 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.472927    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.473391 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.473366    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 15:41:45.475377 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.474289    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9mvsp\""
Sep 10 15:41:45.475898 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.475511    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.477977 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.477961    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.480324 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.480299    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.482797 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.482779    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.485293 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.485278    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:45.485381 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.485362    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:41:45.486645 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.486631    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 15:41:45.487691 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.487672    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:41:45.487795 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.487758    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 15:41:45.487852 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.487834    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 15:41:45.487908 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.487890    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 15:41:45.487991 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.487972    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 15:41:45.488047 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488008    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 15:41:45.488107 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488095    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 15:41:45.488235 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488221    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mzpl6\""
Sep 10 15:41:45.488235 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488227    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7cmh2\""
Sep 10 15:41:45.488355 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488281    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 15:41:45.488355 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488336    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 15:41:45.488456 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488384    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 15:41:45.488526 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488512    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 15:41:45.488588 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488526    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 15:41:45.488588 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488535    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 15:41:45.488691 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488680    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.488744 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488710    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 15:41:45.488881 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.488865    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 15:41:45.489042 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.489027    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4d5k2\""
Sep 10 15:41:45.489146 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.489132    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 15:41:45.489252 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.489238    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mww2g\""
Sep 10 15:41:45.489324 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.489270    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:41:45.489439 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.489416    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zxhxk\""
Sep 10 15:41:45.489814 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.489798    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 15:41:45.490756 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.490740    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.493220 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.493168    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.496904 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.496885    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:45.497001 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.496971    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:41:45.497693 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.497671    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:45.497810 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.497630    2527 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 15:36:44 +0000 UTC" deadline="2028-03-22 02:18:37.776446756 +0000 UTC"
Sep 10 15:41:45.497810 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.497747    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:41:45.497810 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.497761    2527 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13402h36m52.27869065s"
Sep 10 15:41:45.501495 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.501468    2527 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 15:41:45.501624 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.501500    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 15:41:45.502044 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.502028    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 15:41:45.502206 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.502164    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 15:41:45.502378 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.502361    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b6cbt\""
Sep 10 15:41:45.503239 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.503216    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 15:41:45.506082 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.506066    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 15:41:45.506346 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.506330    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 15:41:45.506954 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.506940    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n7hmt\""
Sep 10 15:41:45.507337 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.507323    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bg4rj\""
Sep 10 15:41:45.517781 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.517748    2527 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pnkq7"
Sep 10 15:41:45.529991 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.529968    2527 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pnkq7"
Sep 10 15:41:45.544618 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.544586    2527 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:41:45.567455 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.567420    2527 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 15:41:45.572552 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572529    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbfll\" (UniqueName: \"kubernetes.io/projected/193f19ff-ce68-42c7-929f-f26b2c5edde7-kube-api-access-tbfll\") pod \"node-ca-xfqhc\" (UID: \"193f19ff-ce68-42c7-929f-f26b2c5edde7\") " pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.572631 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572564    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/092a8de9-5d9f-4c7e-8bc3-b56710f67145-ovnkube-config\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.572631 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572585    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bzvv\" (UniqueName: \"kubernetes.io/projected/54d5a997-3142-4442-a345-298d70f46804-kube-api-access-2bzvv\") pod \"node-resolver-4djmx\" (UID: \"54d5a997-3142-4442-a345-298d70f46804\") " pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.572631 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572622    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40c2ed6-ebbb-4c90-919c-23172dca982e-system-cni-dir\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.572760 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572639    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-run-netns\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.572760 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572658    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-modprobe-d\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.572760 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572681    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-log-socket\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.572760 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572724    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-run-ovn-kubernetes\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.572760 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572750    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.572915 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572771    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-sys-fs\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.572915 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572800    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40c2ed6-ebbb-4c90-919c-23172dca982e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.572915 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572818    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-socket-dir-parent\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.572915 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572840    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-run-k8s-cni-cncf-io\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.572915 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572857    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-sys\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.572915 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572886    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16678c3a-9084-48bd-8754-b52212f921f2-etc-tuned\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.573095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572916    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:45.573095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572937    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.573095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.572971    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/80328abc-4881-4a8e-a49e-0bd2abcd7edd-iptables-alerter-script\") pod \"iptables-alerter-h9h7f\" (UID: \"80328abc-4881-4a8e-a49e-0bd2abcd7edd\") " pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.573095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573009    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80328abc-4881-4a8e-a49e-0bd2abcd7edd-host-slash\") pod \"iptables-alerter-h9h7f\" (UID: \"80328abc-4881-4a8e-a49e-0bd2abcd7edd\") " pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.573095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573039    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-var-lib-cni-bin\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.573095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573060    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-var-lib-kubelet\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.573095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573076    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-sysctl-conf\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573092    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-cni-bin\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573131    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-socket-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573165    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-var-lib-cni-multus\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573196    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-conf-dir\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573215    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-run-multus-certs\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573236    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-etc-kubernetes\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573252    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6922\" (UniqueName: \"kubernetes.io/projected/3bf9a25e-613d-4f16-8e38-79be9d27bdff-kube-api-access-d6922\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573268    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-kubelet\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573287    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-systemd-units\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573304    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-slash\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573322    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-run-ovn\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573337    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-node-log\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573366    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/092a8de9-5d9f-4c7e-8bc3-b56710f67145-env-overrides\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573387    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40c2ed6-ebbb-4c90-919c-23172dca982e-cnibin\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573415    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-hostroot\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573444    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-run-systemd\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573467    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-var-lib-openvswitch\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573493    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573510    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-var-lib-kubelet\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573528    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-device-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573551    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvdfp\" (UniqueName: \"kubernetes.io/projected/a40c2ed6-ebbb-4c90-919c-23172dca982e-kube-api-access-zvdfp\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573571    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-run-openvswitch\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573593    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16678c3a-9084-48bd-8754-b52212f921f2-tmp\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573622    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sqbs\" (UniqueName: \"kubernetes.io/projected/16678c3a-9084-48bd-8754-b52212f921f2-kube-api-access-6sqbs\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573640    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f0d7f839-f1d5-4657-aa6c-5b19b248622b-kubelet-config\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:45.573657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573658    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca288a4f-6d91-475a-bdd0-22fb9c6c8361-agent-certs\") pod \"konnectivity-agent-v96j6\" (UID: \"ca288a4f-6d91-475a-bdd0-22fb9c6c8361\") " pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573674    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/193f19ff-ce68-42c7-929f-f26b2c5edde7-host\") pod \"node-ca-xfqhc\" (UID: \"193f19ff-ce68-42c7-929f-f26b2c5edde7\") " pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573691    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxcf6\" (UniqueName: \"kubernetes.io/projected/092a8de9-5d9f-4c7e-8bc3-b56710f67145-kube-api-access-lxcf6\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573708    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a40c2ed6-ebbb-4c90-919c-23172dca982e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573724    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a40c2ed6-ebbb-4c90-919c-23172dca982e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573742    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-system-cni-dir\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573763    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-cnibin\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573778    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-daemon-config\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573836    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-sysconfig\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573889    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-kubernetes\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573911    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573947    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54d5a997-3142-4442-a345-298d70f46804-hosts-file\") pod \"node-resolver-4djmx\" (UID: \"54d5a997-3142-4442-a345-298d70f46804\") " pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573971    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-cni-dir\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.573987    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3bf9a25e-613d-4f16-8e38-79be9d27bdff-cni-binary-copy\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574006    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-lib-modules\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574029    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/193f19ff-ce68-42c7-929f-f26b2c5edde7-serviceca\") pod \"node-ca-xfqhc\" (UID: \"193f19ff-ce68-42c7-929f-f26b2c5edde7\") " pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574045    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-host\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.574270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574086    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-run-netns\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574122    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf5pt\" (UniqueName: \"kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt\") pod \"network-check-target-qswkd\" (UID: \"332552b5-8c5f-4050-81f0-cf2d5377dd09\") " pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574163    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-etc-openvswitch\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574230    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/092a8de9-5d9f-4c7e-8bc3-b56710f67145-ovn-node-metrics-cert\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574259    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/092a8de9-5d9f-4c7e-8bc3-b56710f67145-ovnkube-script-lib\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574282    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-registration-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574315    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpz55\" (UniqueName: \"kubernetes.io/projected/b73aa4cb-f1d1-4166-84e6-224ab92702e2-kube-api-access-dpz55\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574341    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-cni-netd\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574366    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f0d7f839-f1d5-4657-aa6c-5b19b248622b-dbus\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574400    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca288a4f-6d91-475a-bdd0-22fb9c6c8361-konnectivity-ca\") pod \"konnectivity-agent-v96j6\" (UID: \"ca288a4f-6d91-475a-bdd0-22fb9c6c8361\") " pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574421    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40c2ed6-ebbb-4c90-919c-23172dca982e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574443    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wsv8\" (UniqueName: \"kubernetes.io/projected/511e3fe0-d13f-413c-9877-1e056560de23-kube-api-access-2wsv8\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574468    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-systemd\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574492    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-run\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574512    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d5a997-3142-4442-a345-298d70f46804-tmp-dir\") pod \"node-resolver-4djmx\" (UID: \"54d5a997-3142-4442-a345-298d70f46804\") " pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574528    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbhvp\" (UniqueName: \"kubernetes.io/projected/80328abc-4881-4a8e-a49e-0bd2abcd7edd-kube-api-access-pbhvp\") pod \"iptables-alerter-h9h7f\" (UID: \"80328abc-4881-4a8e-a49e-0bd2abcd7edd\") " pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.574892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.574574    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-sysctl-d\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.676110 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.676056    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-sysconfig\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.676394 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.676136    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-kubernetes\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.676394 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.676173    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.676394 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.676244    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54d5a997-3142-4442-a345-298d70f46804-hosts-file\") pod \"node-resolver-4djmx\" (UID: \"54d5a997-3142-4442-a345-298d70f46804\") " pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.676394 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.676376    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54d5a997-3142-4442-a345-298d70f46804-hosts-file\") pod \"node-resolver-4djmx\" (UID: \"54d5a997-3142-4442-a345-298d70f46804\") " pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.676634 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.676473    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-sysconfig\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.676692 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.676631    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-kubernetes\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.676750 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.676700    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.677506 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677465    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-cni-dir\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.677647 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677531    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3bf9a25e-613d-4f16-8e38-79be9d27bdff-cni-binary-copy\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.677647 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677563    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-lib-modules\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.677647 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677588    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/193f19ff-ce68-42c7-929f-f26b2c5edde7-serviceca\") pod \"node-ca-xfqhc\" (UID: \"193f19ff-ce68-42c7-929f-f26b2c5edde7\") " pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.677647 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677618    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-host\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.677854 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677650    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-run-netns\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.677854 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677683    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5pt\" (UniqueName: \"kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt\") pod \"network-check-target-qswkd\" (UID: \"332552b5-8c5f-4050-81f0-cf2d5377dd09\") " pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:45.677854 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677716    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-etc-openvswitch\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.677854 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677749    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/092a8de9-5d9f-4c7e-8bc3-b56710f67145-ovn-node-metrics-cert\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.677854 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677793    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/092a8de9-5d9f-4c7e-8bc3-b56710f67145-ovnkube-script-lib\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.677854 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677820    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-registration-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.677854 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677848    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpz55\" (UniqueName: \"kubernetes.io/projected/b73aa4cb-f1d1-4166-84e6-224ab92702e2-kube-api-access-dpz55\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677879    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-cni-netd\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677917    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f0d7f839-f1d5-4657-aa6c-5b19b248622b-dbus\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677952    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca288a4f-6d91-475a-bdd0-22fb9c6c8361-konnectivity-ca\") pod \"konnectivity-agent-v96j6\" (UID: \"ca288a4f-6d91-475a-bdd0-22fb9c6c8361\") " pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.677981    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40c2ed6-ebbb-4c90-919c-23172dca982e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678013    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wsv8\" (UniqueName: \"kubernetes.io/projected/511e3fe0-d13f-413c-9877-1e056560de23-kube-api-access-2wsv8\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678047    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-systemd\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678071    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-run\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678103    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d5a997-3142-4442-a345-298d70f46804-tmp-dir\") pod \"node-resolver-4djmx\" (UID: \"54d5a997-3142-4442-a345-298d70f46804\") " pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678136    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbhvp\" (UniqueName: \"kubernetes.io/projected/80328abc-4881-4a8e-a49e-0bd2abcd7edd-kube-api-access-pbhvp\") pod \"iptables-alerter-h9h7f\" (UID: \"80328abc-4881-4a8e-a49e-0bd2abcd7edd\") " pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678170    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-sysctl-d\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678227    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbfll\" (UniqueName: \"kubernetes.io/projected/193f19ff-ce68-42c7-929f-f26b2c5edde7-kube-api-access-tbfll\") pod \"node-ca-xfqhc\" (UID: \"193f19ff-ce68-42c7-929f-f26b2c5edde7\") " pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.678271 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678255    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/092a8de9-5d9f-4c7e-8bc3-b56710f67145-ovnkube-config\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678289    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bzvv\" (UniqueName: \"kubernetes.io/projected/54d5a997-3142-4442-a345-298d70f46804-kube-api-access-2bzvv\") pod \"node-resolver-4djmx\" (UID: \"54d5a997-3142-4442-a345-298d70f46804\") " pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678321    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40c2ed6-ebbb-4c90-919c-23172dca982e-system-cni-dir\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678354    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-run-netns\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678383    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-modprobe-d\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678432    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-log-socket\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678501    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-run-ovn-kubernetes\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678518    2527 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678586    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678631    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-sys-fs\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678659    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40c2ed6-ebbb-4c90-919c-23172dca982e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678693    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-socket-dir-parent\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678738    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-run-k8s-cni-cncf-io\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678787    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-sys\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.678840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678820    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16678c3a-9084-48bd-8754-b52212f921f2-etc-tuned\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678853    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678884    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678913    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/80328abc-4881-4a8e-a49e-0bd2abcd7edd-iptables-alerter-script\") pod \"iptables-alerter-h9h7f\" (UID: \"80328abc-4881-4a8e-a49e-0bd2abcd7edd\") " pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678944    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80328abc-4881-4a8e-a49e-0bd2abcd7edd-host-slash\") pod \"iptables-alerter-h9h7f\" (UID: \"80328abc-4881-4a8e-a49e-0bd2abcd7edd\") " pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.678976    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-var-lib-cni-bin\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679010    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-var-lib-kubelet\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679039    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-sysctl-conf\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679069    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-cni-bin\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679101    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-socket-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679131    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-var-lib-cni-multus\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679166    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-conf-dir\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679193    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-run-multus-certs\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.679509 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679294    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-run-multus-certs\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679705    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/193f19ff-ce68-42c7-929f-f26b2c5edde7-serviceca\") pod \"node-ca-xfqhc\" (UID: \"193f19ff-ce68-42c7-929f-f26b2c5edde7\") " pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679793    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-etc-kubernetes\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679840    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6922\" (UniqueName: \"kubernetes.io/projected/3bf9a25e-613d-4f16-8e38-79be9d27bdff-kube-api-access-d6922\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679882    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-kubelet\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679915    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-systemd-units\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679951    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-slash\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.679954    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/092a8de9-5d9f-4c7e-8bc3-b56710f67145-ovnkube-script-lib\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680019    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-slash\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680036    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-registration-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.680196 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680173    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-run-ovn\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680231    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-node-log\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680267    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/092a8de9-5d9f-4c7e-8bc3-b56710f67145-env-overrides\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680302    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40c2ed6-ebbb-4c90-919c-23172dca982e-cnibin\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.680688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680330    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-hostroot\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.680688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680363    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-run-systemd\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680465    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-var-lib-openvswitch\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680482    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-run-systemd\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680547    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-var-lib-openvswitch\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.680688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680642    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-cni-dir\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.681140 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680872    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-cni-netd\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.681140 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680918    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-sys-fs\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.681140 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680970    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f0d7f839-f1d5-4657-aa6c-5b19b248622b-dbus\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:45.681140 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681017    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/092a8de9-5d9f-4c7e-8bc3-b56710f67145-env-overrides\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.681140 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681038    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-socket-dir-parent\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.681140 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681091    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-hostroot\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.681140 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681110    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-sys\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.681140 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681140    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680384    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-run-ovn-kubernetes\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681238    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-systemd-units\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681265    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-host\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681265    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-node-log\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681320    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-var-lib-kubelet\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681326    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3bf9a25e-613d-4f16-8e38-79be9d27bdff-cni-binary-copy\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681185    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-run-k8s-cni-cncf-io\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681417    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a40c2ed6-ebbb-4c90-919c-23172dca982e-cnibin\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681424    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-var-lib-kubelet\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681467    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-device-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681501    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvdfp\" (UniqueName: \"kubernetes.io/projected/a40c2ed6-ebbb-4c90-919c-23172dca982e-kube-api-access-zvdfp\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681529    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-lib-modules\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681546    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-run-openvswitch\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.681570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681549    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-etc-kubernetes\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681656    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-kubelet\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.681767    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.681881    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs podName:511e3fe0-d13f-413c-9877-1e056560de23 nodeName:}" failed. No retries permitted until 2026-09-10 15:41:46.181826373 +0000 UTC m=+2.097951250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs") pod "network-metrics-daemon-5c6r7" (UID: "511e3fe0-d13f-413c-9877-1e056560de23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680931    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-run-ovn\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.681980    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-run-netns\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.682026    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682036    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-etc-openvswitch\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682037    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ca288a4f-6d91-475a-bdd0-22fb9c6c8361-konnectivity-ca\") pod \"konnectivity-agent-v96j6\" (UID: \"ca288a4f-6d91-475a-bdd0-22fb9c6c8361\") " pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.682087    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret podName:f0d7f839-f1d5-4657-aa6c-5b19b248622b nodeName:}" failed. No retries permitted until 2026-09-10 15:41:46.182069549 +0000 UTC m=+2.098194409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret") pod "global-pull-secret-syncer-z9ct5" (UID: "f0d7f839-f1d5-4657-aa6c-5b19b248622b") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682118    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16678c3a-9084-48bd-8754-b52212f921f2-tmp\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682133    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-run-openvswitch\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682150    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sqbs\" (UniqueName: \"kubernetes.io/projected/16678c3a-9084-48bd-8754-b52212f921f2-kube-api-access-6sqbs\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682187    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f0d7f839-f1d5-4657-aa6c-5b19b248622b-kubelet-config\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682223    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-device-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682279    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca288a4f-6d91-475a-bdd0-22fb9c6c8361-agent-certs\") pod \"konnectivity-agent-v96j6\" (UID: \"ca288a4f-6d91-475a-bdd0-22fb9c6c8361\") " pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:41:45.682307 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682318    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/193f19ff-ce68-42c7-929f-f26b2c5edde7-host\") pod \"node-ca-xfqhc\" (UID: \"193f19ff-ce68-42c7-929f-f26b2c5edde7\") " pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682352    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxcf6\" (UniqueName: \"kubernetes.io/projected/092a8de9-5d9f-4c7e-8bc3-b56710f67145-kube-api-access-lxcf6\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682405    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a40c2ed6-ebbb-4c90-919c-23172dca982e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682483    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a40c2ed6-ebbb-4c90-919c-23172dca982e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682497    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-etc-selinux\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682510    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-var-lib-cni-bin\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682539    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80328abc-4881-4a8e-a49e-0bd2abcd7edd-host-slash\") pod \"iptables-alerter-h9h7f\" (UID: \"80328abc-4881-4a8e-a49e-0bd2abcd7edd\") " pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682421    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-sysctl-conf\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682587    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f0d7f839-f1d5-4657-aa6c-5b19b248622b-kubelet-config\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682592    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-run\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.682601    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/193f19ff-ce68-42c7-929f-f26b2c5edde7-host\") pod \"node-ca-xfqhc\" (UID: \"193f19ff-ce68-42c7-929f-f26b2c5edde7\") " pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683117    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a40c2ed6-ebbb-4c90-919c-23172dca982e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683164    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-system-cni-dir\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683197    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-cnibin\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683291    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-cnibin\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683400    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-daemon-config\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683508    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/092a8de9-5d9f-4c7e-8bc3-b56710f67145-ovnkube-config\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.685410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683508    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-systemd\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683553    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a40c2ed6-ebbb-4c90-919c-23172dca982e-system-cni-dir\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683559    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-modprobe-d\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683612    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-run-netns\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683975    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/16678c3a-9084-48bd-8754-b52212f921f2-etc-sysctl-d\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.683999    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-daemon-config\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.684109    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a40c2ed6-ebbb-4c90-919c-23172dca982e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.684239    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-var-lib-kubelet\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.680969    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.684358    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-log-socket\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.684385    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a40c2ed6-ebbb-4c90-919c-23172dca982e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.684406    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-host-var-lib-cni-multus\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.684499    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-system-cni-dir\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.684538    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b73aa4cb-f1d1-4166-84e6-224ab92702e2-socket-dir\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.684576    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3bf9a25e-613d-4f16-8e38-79be9d27bdff-multus-conf-dir\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.685546    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/80328abc-4881-4a8e-a49e-0bd2abcd7edd-iptables-alerter-script\") pod \"iptables-alerter-h9h7f\" (UID: \"80328abc-4881-4a8e-a49e-0bd2abcd7edd\") " pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.685732    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/092a8de9-5d9f-4c7e-8bc3-b56710f67145-host-cni-bin\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.686189 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.686096    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/092a8de9-5d9f-4c7e-8bc3-b56710f67145-ovn-node-metrics-cert\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.687006 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.686133    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a40c2ed6-ebbb-4c90-919c-23172dca982e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.687006 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.686348    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16678c3a-9084-48bd-8754-b52212f921f2-tmp\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.687006 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.686479    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d5a997-3142-4442-a345-298d70f46804-tmp-dir\") pod \"node-resolver-4djmx\" (UID: \"54d5a997-3142-4442-a345-298d70f46804\") " pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.687246 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.687005    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/16678c3a-9084-48bd-8754-b52212f921f2-etc-tuned\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.687246 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.687219    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ca288a4f-6d91-475a-bdd0-22fb9c6c8361-agent-certs\") pod \"konnectivity-agent-v96j6\" (UID: \"ca288a4f-6d91-475a-bdd0-22fb9c6c8361\") " pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:41:45.692033 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.689332    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpz55\" (UniqueName: \"kubernetes.io/projected/b73aa4cb-f1d1-4166-84e6-224ab92702e2-kube-api-access-dpz55\") pod \"aws-ebs-csi-driver-node-lbn65\" (UID: \"b73aa4cb-f1d1-4166-84e6-224ab92702e2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.692033 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.690233    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wsv8\" (UniqueName: \"kubernetes.io/projected/511e3fe0-d13f-413c-9877-1e056560de23-kube-api-access-2wsv8\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:45.692033 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.690396    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:41:45.692033 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.690416    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:41:45.692033 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.690431    2527 projected.go:196] Error preparing data for projected volume kube-api-access-mf5pt for pod openshift-network-diagnostics/network-check-target-qswkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:45.692033 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.690506    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt podName:332552b5-8c5f-4050-81f0-cf2d5377dd09 nodeName:}" failed. No retries permitted until 2026-09-10 15:41:46.190486043 +0000 UTC m=+2.106610921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mf5pt" (UniqueName: "kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt") pod "network-check-target-qswkd" (UID: "332552b5-8c5f-4050-81f0-cf2d5377dd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:45.692033 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.691967    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvdfp\" (UniqueName: \"kubernetes.io/projected/a40c2ed6-ebbb-4c90-919c-23172dca982e-kube-api-access-zvdfp\") pod \"multus-additional-cni-plugins-ps5pl\" (UID: \"a40c2ed6-ebbb-4c90-919c-23172dca982e\") " pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.692466 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.692051    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxcf6\" (UniqueName: \"kubernetes.io/projected/092a8de9-5d9f-4c7e-8bc3-b56710f67145-kube-api-access-lxcf6\") pod \"ovnkube-node-zb2zb\" (UID: \"092a8de9-5d9f-4c7e-8bc3-b56710f67145\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.692466 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.692318    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbhvp\" (UniqueName: \"kubernetes.io/projected/80328abc-4881-4a8e-a49e-0bd2abcd7edd-kube-api-access-pbhvp\") pod \"iptables-alerter-h9h7f\" (UID: \"80328abc-4881-4a8e-a49e-0bd2abcd7edd\") " pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.692466 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.692406    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbfll\" (UniqueName: \"kubernetes.io/projected/193f19ff-ce68-42c7-929f-f26b2c5edde7-kube-api-access-tbfll\") pod \"node-ca-xfqhc\" (UID: \"193f19ff-ce68-42c7-929f-f26b2c5edde7\") " pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.694552 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.694522    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sqbs\" (UniqueName: \"kubernetes.io/projected/16678c3a-9084-48bd-8754-b52212f921f2-kube-api-access-6sqbs\") pod \"tuned-s2xpk\" (UID: \"16678c3a-9084-48bd-8754-b52212f921f2\") " pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.694821 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.694800    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6922\" (UniqueName: \"kubernetes.io/projected/3bf9a25e-613d-4f16-8e38-79be9d27bdff-kube-api-access-d6922\") pod \"multus-sv2hv\" (UID: \"3bf9a25e-613d-4f16-8e38-79be9d27bdff\") " pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.695327 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.695310    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bzvv\" (UniqueName: \"kubernetes.io/projected/54d5a997-3142-4442-a345-298d70f46804-kube-api-access-2bzvv\") pod \"node-resolver-4djmx\" (UID: \"54d5a997-3142-4442-a345-298d70f46804\") " pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.790069 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.790032    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a61b7f916a5c74ee03e2f7481d45ef.slice/crio-e05a63cebac46746679a98de1ed0f9fd7366e1290a538d9e4cb99e9d52f4497d WatchSource:0}: Error finding container e05a63cebac46746679a98de1ed0f9fd7366e1290a538d9e4cb99e9d52f4497d: Status 404 returned error can't find the container with id e05a63cebac46746679a98de1ed0f9fd7366e1290a538d9e4cb99e9d52f4497d
Sep 10 15:41:45.798177 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.798128    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:41:45.801129 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.801104    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:41:45.801947 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.801923    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c9407f7c27661a1f8778abcf84b0b26.slice/crio-7c4a0b364a6dee0bb8f4c9f002f41aea0c6a77e17328209482b12019083c7cd7 WatchSource:0}: Error finding container 7c4a0b364a6dee0bb8f4c9f002f41aea0c6a77e17328209482b12019083c7cd7: Status 404 returned error can't find the container with id 7c4a0b364a6dee0bb8f4c9f002f41aea0c6a77e17328209482b12019083c7cd7
Sep 10 15:41:45.812850 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.812819    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca288a4f_6d91_475a_bdd0_22fb9c6c8361.slice/crio-f6a0d95f234c9571e2bebf2aaec4a78f0097511a662c60a2469189677d93f2a9 WatchSource:0}: Error finding container f6a0d95f234c9571e2bebf2aaec4a78f0097511a662c60a2469189677d93f2a9: Status 404 returned error can't find the container with id f6a0d95f234c9571e2bebf2aaec4a78f0097511a662c60a2469189677d93f2a9
Sep 10 15:41:45.819793 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.819767    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s2xpk"
Sep 10 15:41:45.828788 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.828761    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16678c3a_9084_48bd_8754_b52212f921f2.slice/crio-f82f9713451ddec8e9601261ee409a6b757a183043856d0e86cc1eec0e5215ab WatchSource:0}: Error finding container f82f9713451ddec8e9601261ee409a6b757a183043856d0e86cc1eec0e5215ab: Status 404 returned error can't find the container with id f82f9713451ddec8e9601261ee409a6b757a183043856d0e86cc1eec0e5215ab
Sep 10 15:41:45.837551 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.837528    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xfqhc"
Sep 10 15:41:45.844858 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.844828    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ps5pl"
Sep 10 15:41:45.856736 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.856698    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40c2ed6_ebbb_4c90_919c_23172dca982e.slice/crio-4cb7f06c52dd7fc1cba306f82cc109ac2576c10fb39b086801c9a92a2758be31 WatchSource:0}: Error finding container 4cb7f06c52dd7fc1cba306f82cc109ac2576c10fb39b086801c9a92a2758be31: Status 404 returned error can't find the container with id 4cb7f06c52dd7fc1cba306f82cc109ac2576c10fb39b086801c9a92a2758be31
Sep 10 15:41:45.872395 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.872362    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h9h7f"
Sep 10 15:41:45.881588 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.881557    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80328abc_4881_4a8e_a49e_0bd2abcd7edd.slice/crio-dec7a1297706e13e08cf5aeaef61e06e5ab4daa6e48665637f6524123c71dde3 WatchSource:0}: Error finding container dec7a1297706e13e08cf5aeaef61e06e5ab4daa6e48665637f6524123c71dde3: Status 404 returned error can't find the container with id dec7a1297706e13e08cf5aeaef61e06e5ab4daa6e48665637f6524123c71dde3
Sep 10 15:41:45.885016 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.884990    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:41:45.895095 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.895061    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod092a8de9_5d9f_4c7e_8bc3_b56710f67145.slice/crio-ae996b55658aeacec1426c7e751dc87de8bbbc73c4e494433cd5dd93a777dd34 WatchSource:0}: Error finding container ae996b55658aeacec1426c7e751dc87de8bbbc73c4e494433cd5dd93a777dd34: Status 404 returned error can't find the container with id ae996b55658aeacec1426c7e751dc87de8bbbc73c4e494433cd5dd93a777dd34
Sep 10 15:41:45.903944 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.903916    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65"
Sep 10 15:41:45.913461 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.913427    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73aa4cb_f1d1_4166_84e6_224ab92702e2.slice/crio-41eb3deb3eb778a8d4988962d18c8b43a9b95a2302b51d4a9dd1e8ed04e521b4 WatchSource:0}: Error finding container 41eb3deb3eb778a8d4988962d18c8b43a9b95a2302b51d4a9dd1e8ed04e521b4: Status 404 returned error can't find the container with id 41eb3deb3eb778a8d4988962d18c8b43a9b95a2302b51d4a9dd1e8ed04e521b4
Sep 10 15:41:45.928364 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.928337    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4djmx"
Sep 10 15:41:45.933646 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:45.933624    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sv2hv"
Sep 10 15:41:45.938182 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.938151    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d5a997_3142_4442_a345_298d70f46804.slice/crio-0364ba65e3f9bcabd4ca6aac67c7bef32b2d37b2300554c3a0c454a567da15f5 WatchSource:0}: Error finding container 0364ba65e3f9bcabd4ca6aac67c7bef32b2d37b2300554c3a0c454a567da15f5: Status 404 returned error can't find the container with id 0364ba65e3f9bcabd4ca6aac67c7bef32b2d37b2300554c3a0c454a567da15f5
Sep 10 15:41:45.943872 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:41:45.943847    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf9a25e_613d_4f16_8e38_79be9d27bdff.slice/crio-9b0884d74701d3bf9bdc7526a5ada5a20dc846e76291fcf0f69827bda9159ae1 WatchSource:0}: Error finding container 9b0884d74701d3bf9bdc7526a5ada5a20dc846e76291fcf0f69827bda9159ae1: Status 404 returned error can't find the container with id 9b0884d74701d3bf9bdc7526a5ada5a20dc846e76291fcf0f69827bda9159ae1
Sep 10 15:41:45.945817 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.945786    2527 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-sv2hv_openshift-multus(3bf9a25e-613d-4f16-8e38-79be9d27bdff): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 10 15:41:45.945904 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:45.945834    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-sv2hv" podUID="3bf9a25e-613d-4f16-8e38-79be9d27bdff"
Sep 10 15:41:46.188048 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.187951    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:46.188048 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.188001    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:46.188254 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.188116    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:46.188254 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.188204    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret podName:f0d7f839-f1d5-4657-aa6c-5b19b248622b nodeName:}" failed. No retries permitted until 2026-09-10 15:41:47.188186637 +0000 UTC m=+3.104311499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret") pod "global-pull-secret-syncer-z9ct5" (UID: "f0d7f839-f1d5-4657-aa6c-5b19b248622b") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:46.188254 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.188120    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:46.188420 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.188260    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs podName:511e3fe0-d13f-413c-9877-1e056560de23 nodeName:}" failed. No retries permitted until 2026-09-10 15:41:47.188244659 +0000 UTC m=+3.104369532 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs") pod "network-metrics-daemon-5c6r7" (UID: "511e3fe0-d13f-413c-9877-1e056560de23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:46.289167 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.289126    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5pt\" (UniqueName: \"kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt\") pod \"network-check-target-qswkd\" (UID: \"332552b5-8c5f-4050-81f0-cf2d5377dd09\") " pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:46.289445 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.289393    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:41:46.289491 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.289457    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:41:46.289491 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.289474    2527 projected.go:196] Error preparing data for projected volume kube-api-access-mf5pt for pod openshift-network-diagnostics/network-check-target-qswkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:46.289611 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.289549    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt podName:332552b5-8c5f-4050-81f0-cf2d5377dd09 nodeName:}" failed. No retries permitted until 2026-09-10 15:41:47.289528387 +0000 UTC m=+3.205653252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf5pt" (UniqueName: "kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt") pod "network-check-target-qswkd" (UID: "332552b5-8c5f-4050-81f0-cf2d5377dd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:46.533522 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.533339    2527 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:36:45 +0000 UTC" deadline="2028-03-29 02:02:34.485075965 +0000 UTC"
Sep 10 15:41:46.533522 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.533416    2527 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13570h20m47.951665478s"
Sep 10 15:41:46.569051 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.569015    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:46.569272 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.569158    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:41:46.574931 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.574741    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s2xpk" event={"ID":"16678c3a-9084-48bd-8754-b52212f921f2","Type":"ContainerStarted","Data":"f82f9713451ddec8e9601261ee409a6b757a183043856d0e86cc1eec0e5215ab"}
Sep 10 15:41:46.584934 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.584877    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal" event={"ID":"72a61b7f916a5c74ee03e2f7481d45ef","Type":"ContainerStarted","Data":"e05a63cebac46746679a98de1ed0f9fd7366e1290a538d9e4cb99e9d52f4497d"}
Sep 10 15:41:46.590521 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.590480    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sv2hv" event={"ID":"3bf9a25e-613d-4f16-8e38-79be9d27bdff","Type":"ContainerStarted","Data":"9b0884d74701d3bf9bdc7526a5ada5a20dc846e76291fcf0f69827bda9159ae1"}
Sep 10 15:41:46.602646 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:46.602603    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-sv2hv" podUID="3bf9a25e-613d-4f16-8e38-79be9d27bdff"
Sep 10 15:41:46.617573 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.617450    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65" event={"ID":"b73aa4cb-f1d1-4166-84e6-224ab92702e2","Type":"ContainerStarted","Data":"41eb3deb3eb778a8d4988962d18c8b43a9b95a2302b51d4a9dd1e8ed04e521b4"}
Sep 10 15:41:46.624461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.624416    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" event={"ID":"092a8de9-5d9f-4c7e-8bc3-b56710f67145","Type":"ContainerStarted","Data":"ae996b55658aeacec1426c7e751dc87de8bbbc73c4e494433cd5dd93a777dd34"}
Sep 10 15:41:46.636932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.636888    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v96j6" event={"ID":"ca288a4f-6d91-475a-bdd0-22fb9c6c8361","Type":"ContainerStarted","Data":"f6a0d95f234c9571e2bebf2aaec4a78f0097511a662c60a2469189677d93f2a9"}
Sep 10 15:41:46.640260 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.640223    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal" event={"ID":"9c9407f7c27661a1f8778abcf84b0b26","Type":"ContainerStarted","Data":"7c4a0b364a6dee0bb8f4c9f002f41aea0c6a77e17328209482b12019083c7cd7"}
Sep 10 15:41:46.642365 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.642329    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4djmx" event={"ID":"54d5a997-3142-4442-a345-298d70f46804","Type":"ContainerStarted","Data":"0364ba65e3f9bcabd4ca6aac67c7bef32b2d37b2300554c3a0c454a567da15f5"}
Sep 10 15:41:46.647504 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.647464    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h9h7f" event={"ID":"80328abc-4881-4a8e-a49e-0bd2abcd7edd","Type":"ContainerStarted","Data":"dec7a1297706e13e08cf5aeaef61e06e5ab4daa6e48665637f6524123c71dde3"}
Sep 10 15:41:46.659039 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.658995    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps5pl" event={"ID":"a40c2ed6-ebbb-4c90-919c-23172dca982e","Type":"ContainerStarted","Data":"4cb7f06c52dd7fc1cba306f82cc109ac2576c10fb39b086801c9a92a2758be31"}
Sep 10 15:41:46.664578 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.664535    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xfqhc" event={"ID":"193f19ff-ce68-42c7-929f-f26b2c5edde7","Type":"ContainerStarted","Data":"b23e3bc6de29dcceac23a0369158810aa8e408f765458a808004a564f26da694"}
Sep 10 15:41:46.698106 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:46.698071    2527 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:41:47.197149 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:47.197031    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:47.197149 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:47.197104    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:47.197536 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.197294    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:47.197536 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.197364    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs podName:511e3fe0-d13f-413c-9877-1e056560de23 nodeName:}" failed. No retries permitted until 2026-09-10 15:41:49.197343093 +0000 UTC m=+5.113467957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs") pod "network-metrics-daemon-5c6r7" (UID: "511e3fe0-d13f-413c-9877-1e056560de23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:47.198007 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.197833    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:47.198007 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.197903    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret podName:f0d7f839-f1d5-4657-aa6c-5b19b248622b nodeName:}" failed. No retries permitted until 2026-09-10 15:41:49.197883432 +0000 UTC m=+5.114008314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret") pod "global-pull-secret-syncer-z9ct5" (UID: "f0d7f839-f1d5-4657-aa6c-5b19b248622b") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:47.297615 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:47.297567    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5pt\" (UniqueName: \"kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt\") pod \"network-check-target-qswkd\" (UID: \"332552b5-8c5f-4050-81f0-cf2d5377dd09\") " pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:47.297830 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.297800    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:41:47.297830 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.297825    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:41:47.297942 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.297841    2527 projected.go:196] Error preparing data for projected volume kube-api-access-mf5pt for pod openshift-network-diagnostics/network-check-target-qswkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:47.297942 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.297907    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt podName:332552b5-8c5f-4050-81f0-cf2d5377dd09 nodeName:}" failed. No retries permitted until 2026-09-10 15:41:49.29788732 +0000 UTC m=+5.214012199 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf5pt" (UniqueName: "kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt") pod "network-check-target-qswkd" (UID: "332552b5-8c5f-4050-81f0-cf2d5377dd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:47.534478 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:47.534342    2527 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:36:45 +0000 UTC" deadline="2028-05-13 20:12:03.723201863 +0000 UTC"
Sep 10 15:41:47.534478 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:47.534418    2527 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14668h30m16.188788264s"
Sep 10 15:41:47.559548 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:47.559509    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:47.559763 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.559700    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:41:47.560359 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:47.560165    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:47.560359 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:47.560303    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:41:48.019592 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:48.019472    2527 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 15:41:48.561354 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:48.561307    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:48.561920 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:48.561463    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:41:49.214709 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:49.213771    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:49.214709 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:49.213845    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:49.214709 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.213993    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:49.214709 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.214066    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs podName:511e3fe0-d13f-413c-9877-1e056560de23 nodeName:}" failed. No retries permitted until 2026-09-10 15:41:53.214043819 +0000 UTC m=+9.130168684 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs") pod "network-metrics-daemon-5c6r7" (UID: "511e3fe0-d13f-413c-9877-1e056560de23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:49.214709 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.214544    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:49.214709 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.214609    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret podName:f0d7f839-f1d5-4657-aa6c-5b19b248622b nodeName:}" failed. No retries permitted until 2026-09-10 15:41:53.2145917 +0000 UTC m=+9.130716563 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret") pod "global-pull-secret-syncer-z9ct5" (UID: "f0d7f839-f1d5-4657-aa6c-5b19b248622b") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:49.314662 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:49.314617    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5pt\" (UniqueName: \"kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt\") pod \"network-check-target-qswkd\" (UID: \"332552b5-8c5f-4050-81f0-cf2d5377dd09\") " pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:49.314867 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.314832    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:41:49.314867 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.314856    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:41:49.314983 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.314870    2527 projected.go:196] Error preparing data for projected volume kube-api-access-mf5pt for pod openshift-network-diagnostics/network-check-target-qswkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:49.314983 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.314935    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt podName:332552b5-8c5f-4050-81f0-cf2d5377dd09 nodeName:}" failed. No retries permitted until 2026-09-10 15:41:53.314915972 +0000 UTC m=+9.231040837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf5pt" (UniqueName: "kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt") pod "network-check-target-qswkd" (UID: "332552b5-8c5f-4050-81f0-cf2d5377dd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:49.558074 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:49.557971    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:49.558304 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.558135    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:41:49.558304 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:49.558281    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:49.558440 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:49.558401    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:41:50.557530 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:50.557496    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:50.558161 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:50.557643    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:41:51.557799 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:51.557760    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:51.557799 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:51.557786    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:51.558390 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:51.557897    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:41:51.558390 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:51.558004    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:41:52.557702 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:52.557670    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:52.557893 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:52.557811    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:41:53.249880 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:53.249843    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:53.250086 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:53.249905    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:53.250086 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.250063    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:53.250242 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.250129    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs podName:511e3fe0-d13f-413c-9877-1e056560de23 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:01.250109834 +0000 UTC m=+17.166234696 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs") pod "network-metrics-daemon-5c6r7" (UID: "511e3fe0-d13f-413c-9877-1e056560de23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:41:53.250242 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.250061    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:53.250242 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.250215    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret podName:f0d7f839-f1d5-4657-aa6c-5b19b248622b nodeName:}" failed. No retries permitted until 2026-09-10 15:42:01.250194175 +0000 UTC m=+17.166319036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret") pod "global-pull-secret-syncer-z9ct5" (UID: "f0d7f839-f1d5-4657-aa6c-5b19b248622b") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:41:53.350661 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:53.350598    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5pt\" (UniqueName: \"kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt\") pod \"network-check-target-qswkd\" (UID: \"332552b5-8c5f-4050-81f0-cf2d5377dd09\") " pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:53.350862 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.350730    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:41:53.350862 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.350754    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:41:53.350862 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.350768    2527 projected.go:196] Error preparing data for projected volume kube-api-access-mf5pt for pod openshift-network-diagnostics/network-check-target-qswkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:53.350862 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.350838    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt podName:332552b5-8c5f-4050-81f0-cf2d5377dd09 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:01.35081723 +0000 UTC m=+17.266942107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf5pt" (UniqueName: "kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt") pod "network-check-target-qswkd" (UID: "332552b5-8c5f-4050-81f0-cf2d5377dd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:41:53.557534 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:53.557488    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:53.557717 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.557634    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:41:53.557717 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:53.557667    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:53.557838 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:53.557756    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:41:54.557873 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:54.557837    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:54.558631 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:54.557957    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:41:55.557796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:55.557751    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:55.557992 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:55.557884    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:41:55.558389 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:55.557771    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:55.558446 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:55.558424    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:41:56.560609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:56.560577    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:56.561028 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:56.560693    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:41:57.557595 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:57.557557    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:57.557796 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:57.557673    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:41:57.557796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:57.557733    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:57.557922 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:57.557867    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:41:58.557913 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:58.557875    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:41:58.559072 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:58.558378    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:41:59.557286 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:59.557247    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:41:59.557286 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:41:59.557270    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:41:59.557516 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:59.557380    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:41:59.557516 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:41:59.557494    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:42:00.558027 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:00.557425    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:00.559836 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:00.559519    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:42:00.721987 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:00.721946    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s2xpk" event={"ID":"16678c3a-9084-48bd-8754-b52212f921f2","Type":"ContainerStarted","Data":"b137942f5834ea77b2825c8fb34d1482393c4d89a6a95ac3badcbc3b42096b1d"}
Sep 10 15:42:00.733316 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:00.733169    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" event={"ID":"092a8de9-5d9f-4c7e-8bc3-b56710f67145","Type":"ContainerStarted","Data":"c2b70b488d5474efb277808d85f07597f3bf621d4cbfadf8b2596f7ec837c887"}
Sep 10 15:42:00.736950 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:00.736901    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal" event={"ID":"9c9407f7c27661a1f8778abcf84b0b26","Type":"ContainerStarted","Data":"c399314ef3e18b06b9597993e1132caa125e12ba49e70607c26ca1f4fa9aa571"}
Sep 10 15:42:00.738205 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:00.738129    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s2xpk" podStartSLOduration=1.341030575 podStartE2EDuration="15.738110166s" totalImagesPullingTime="14.397079591s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:00.736780195 +0000 UTC m=+16.652905097" watchObservedRunningTime="2026-09-10 15:42:00.738110166 +0000 UTC m=+16.654235045"
Sep 10 15:42:00.749403 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:00.749312    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-61.ec2.internal" podStartSLOduration=15.749291958 podStartE2EDuration="15.749291958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:00.748532258 +0000 UTC m=+16.664657156" watchObservedRunningTime="2026-09-10 15:42:00.749291958 +0000 UTC m=+16.665416838"
Sep 10 15:42:01.307612 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.307389    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:01.307855 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.307646    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:01.307855 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.307573    2527 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 15:42:01.307855 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.307748    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:42:01.307855 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.307758    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret podName:f0d7f839-f1d5-4657-aa6c-5b19b248622b nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.307734893 +0000 UTC m=+33.223859836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret") pod "global-pull-secret-syncer-z9ct5" (UID: "f0d7f839-f1d5-4657-aa6c-5b19b248622b") : object "kube-system"/"original-pull-secret" not registered
Sep 10 15:42:01.307855 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.307797    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs podName:511e3fe0-d13f-413c-9877-1e056560de23 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.307782904 +0000 UTC m=+33.223907763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs") pod "network-metrics-daemon-5c6r7" (UID: "511e3fe0-d13f-413c-9877-1e056560de23") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 15:42:01.408564 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.408475    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5pt\" (UniqueName: \"kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt\") pod \"network-check-target-qswkd\" (UID: \"332552b5-8c5f-4050-81f0-cf2d5377dd09\") " pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:01.408741 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.408626    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 15:42:01.408741 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.408647    2527 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 15:42:01.408741 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.408657    2527 projected.go:196] Error preparing data for projected volume kube-api-access-mf5pt for pod openshift-network-diagnostics/network-check-target-qswkd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:42:01.408741 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.408720    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt podName:332552b5-8c5f-4050-81f0-cf2d5377dd09 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.40870034 +0000 UTC m=+33.324825211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf5pt" (UniqueName: "kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt") pod "network-check-target-qswkd" (UID: "332552b5-8c5f-4050-81f0-cf2d5377dd09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 15:42:01.557479 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.557441    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:01.557670 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.557578    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:42:01.557712 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.557675    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:01.557820 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:01.557798    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:42:01.740905 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.740868    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xfqhc" event={"ID":"193f19ff-ce68-42c7-929f-f26b2c5edde7","Type":"ContainerStarted","Data":"18edcb34ba4d6a608076455bcea7f844b17e631441b6a04fe76e7ce09a91437e"}
Sep 10 15:42:01.742713 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.742684    2527 generic.go:356] "Generic (PLEG): container finished" podID="72a61b7f916a5c74ee03e2f7481d45ef" containerID="6a4b4a1d91895b1e82233c3ac6d0df9c446f238c6946779fb9ba70ed7a630e0b" exitCode=0
Sep 10 15:42:01.742858 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.742766    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal" event={"ID":"72a61b7f916a5c74ee03e2f7481d45ef","Type":"ContainerDied","Data":"6a4b4a1d91895b1e82233c3ac6d0df9c446f238c6946779fb9ba70ed7a630e0b"}
Sep 10 15:42:01.744514 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.744489    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65" event={"ID":"b73aa4cb-f1d1-4166-84e6-224ab92702e2","Type":"ContainerStarted","Data":"032d0085d35e940924eb57877455d2eec372f62813d575042df2aa701f3b0ebb"}
Sep 10 15:42:01.747982 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.747956    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" event={"ID":"092a8de9-5d9f-4c7e-8bc3-b56710f67145","Type":"ContainerStarted","Data":"4fce5b5bfc3dc7e86acd864505f886248dacbab8205172904fab10a0618d9ba1"}
Sep 10 15:42:01.748071 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.748024    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" event={"ID":"092a8de9-5d9f-4c7e-8bc3-b56710f67145","Type":"ContainerStarted","Data":"6de7fcbad0ee7bdf6d00f1226b377de867ecf254094c357454e7272339a02d26"}
Sep 10 15:42:01.748149 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.748073    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" event={"ID":"092a8de9-5d9f-4c7e-8bc3-b56710f67145","Type":"ContainerStarted","Data":"de50f38b961a1346b1bad2244c3ace84e23952a03c02c91c2a2d79494a27abc3"}
Sep 10 15:42:01.748149 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.748124    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" event={"ID":"092a8de9-5d9f-4c7e-8bc3-b56710f67145","Type":"ContainerStarted","Data":"a8e414cd14a500d17984d68fb1ab8a054735dc77323cf6801686120602bcf3f8"}
Sep 10 15:42:01.748235 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.748177    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" event={"ID":"092a8de9-5d9f-4c7e-8bc3-b56710f67145","Type":"ContainerStarted","Data":"945dcc2060ae6054db5cd4593aec0f2f4a7087c2604cf4b0160273bb8e9c1567"}
Sep 10 15:42:01.749730 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.749704    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v96j6" event={"ID":"ca288a4f-6d91-475a-bdd0-22fb9c6c8361","Type":"ContainerStarted","Data":"c7c410595d4e7ac8000eeeaae6334d5b2e9b2b3a745600598f97f1d0da05e337"}
Sep 10 15:42:01.751301 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.751276    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4djmx" event={"ID":"54d5a997-3142-4442-a345-298d70f46804","Type":"ContainerStarted","Data":"22545d06b4d96debc743c6c152bca81b3ab9c09d8513c3c3b10a6cc598aec993"}
Sep 10 15:42:01.753120 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.753074    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xfqhc" podStartSLOduration=2.4043102579999998 podStartE2EDuration="16.753058177s" totalImagesPullingTime="14.348747919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:01.752351358 +0000 UTC m=+17.668476238" watchObservedRunningTime="2026-09-10 15:42:01.753058177 +0000 UTC m=+17.669183129"
Sep 10 15:42:01.753231 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.753189    2527 generic.go:356] "Generic (PLEG): container finished" podID="a40c2ed6-ebbb-4c90-919c-23172dca982e" containerID="248d2517bf1f193a49797ed60b6f7dc4d0243bfdd610a595796345e534eecd45" exitCode=0
Sep 10 15:42:01.753283 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.753240    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps5pl" event={"ID":"a40c2ed6-ebbb-4c90-919c-23172dca982e","Type":"ContainerDied","Data":"248d2517bf1f193a49797ed60b6f7dc4d0243bfdd610a595796345e534eecd45"}
Sep 10 15:42:01.779047 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.778980    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4djmx" podStartSLOduration=2.521633399 podStartE2EDuration="16.778947923s" totalImagesPullingTime="14.257314524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:01.778510851 +0000 UTC m=+17.694635734" watchObservedRunningTime="2026-09-10 15:42:01.778947923 +0000 UTC m=+17.695072808"
Sep 10 15:42:01.794569 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:01.794491    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v96j6" podStartSLOduration=6.154783345 podStartE2EDuration="16.794465485s" totalImagesPullingTime="10.63968214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:01.793881922 +0000 UTC m=+17.710006827" watchObservedRunningTime="2026-09-10 15:42:01.794465485 +0000 UTC m=+17.710590366"
Sep 10 15:42:02.020637 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:02.020607    2527 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 15:42:02.546365 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:02.545978    2527 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T15:42:02.020633394Z","UUID":"1f0f1c7d-42e8-49a7-9bff-df967ea75f80","Handler":null,"Name":"","Endpoint":""}
Sep 10 15:42:02.547926 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:02.547896    2527 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 15:42:02.548066 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:02.547938    2527 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 15:42:02.557681 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:02.557649    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:02.557852 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:02.557804    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:42:02.757674 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:02.757635    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal" event={"ID":"72a61b7f916a5c74ee03e2f7481d45ef","Type":"ContainerStarted","Data":"87bd4f99a4dd1e022fd8b1b494aaebb22727e82977491ec4b00a9cff0d6c15b6"}
Sep 10 15:42:02.759985 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:02.759955    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65" event={"ID":"b73aa4cb-f1d1-4166-84e6-224ab92702e2","Type":"ContainerStarted","Data":"fec6f99c3fd19688dcc10c6475cab762add35dd47b9744fb7dbaa7ac844e1aab"}
Sep 10 15:42:02.761807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:02.761769    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h9h7f" event={"ID":"80328abc-4881-4a8e-a49e-0bd2abcd7edd","Type":"ContainerStarted","Data":"b68f5feb196fb74324b40221e38269edf9de8d18fb755071dd1749909e971a48"}
Sep 10 15:42:02.771940 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:02.771890    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-61.ec2.internal" podStartSLOduration=17.77187612 podStartE2EDuration="17.77187612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:02.771503546 +0000 UTC m=+18.687628428" watchObservedRunningTime="2026-09-10 15:42:02.77187612 +0000 UTC m=+18.688001001"
Sep 10 15:42:03.188861 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.188820    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:42:03.388569 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.388538    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:42:03.389369 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.389341    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:42:03.401885 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.401833    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h9h7f" podStartSLOduration=4.08771326 podStartE2EDuration="18.401814419s" totalImagesPullingTime="14.314101159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:02.794439955 +0000 UTC m=+18.710564835" watchObservedRunningTime="2026-09-10 15:42:03.401814419 +0000 UTC m=+19.317939298"
Sep 10 15:42:03.558031 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.557306    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:03.558031 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:03.557451    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:42:03.558031 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.557878    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:03.558031 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:03.557990    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:42:03.769026 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.767856    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65" event={"ID":"b73aa4cb-f1d1-4166-84e6-224ab92702e2","Type":"ContainerStarted","Data":"dfc5a45650c2b0832cefa3fe0a38d870b133404c0d22aafd42fc3c8ecee68a1a"}
Sep 10 15:42:03.774085 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.774010    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" event={"ID":"092a8de9-5d9f-4c7e-8bc3-b56710f67145","Type":"ContainerStarted","Data":"5bd642a0e1a3398100bbf27453281c65062f15c9958af964df215c49dbaba6e3"}
Sep 10 15:42:03.776166 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.775411    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v96j6"
Sep 10 15:42:03.783618 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:03.783567    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lbn65" podStartSLOduration=2.050327148 podStartE2EDuration="18.783549932s" totalImagesPullingTime="16.733222784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:03.783406013 +0000 UTC m=+19.699530893" watchObservedRunningTime="2026-09-10 15:42:03.783549932 +0000 UTC m=+19.699674808"
Sep 10 15:42:04.558606 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:04.558533    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:04.558803 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:04.558660    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:42:05.557918 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:05.557888    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:05.558414 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:05.558010    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:42:05.558414 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:05.558061    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:05.558414 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:05.558219    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:42:06.557702 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:06.557667    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:06.557877 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:06.557803    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:42:07.557630 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:07.557420    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:07.558057 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:07.557467    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:07.558057 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:07.557739    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:42:07.558057 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:07.557817    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:42:07.790715 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:07.790679    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" event={"ID":"092a8de9-5d9f-4c7e-8bc3-b56710f67145","Type":"ContainerStarted","Data":"e397d9601e7e22d389f99ba8b83dbcc697a0c79aa7b4c1b958c8a63a4e2062b4"}
Sep 10 15:42:07.790998 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:07.790930    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:42:07.792805 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:07.792775    2527 generic.go:356] "Generic (PLEG): container finished" podID="a40c2ed6-ebbb-4c90-919c-23172dca982e" containerID="e929eadc08d648f53f8e6e6821b9ab4cceebfcf690614dac0b77dc3c92c8cdca" exitCode=0
Sep 10 15:42:07.792920 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:07.792822    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps5pl" event={"ID":"a40c2ed6-ebbb-4c90-919c-23172dca982e","Type":"ContainerDied","Data":"e929eadc08d648f53f8e6e6821b9ab4cceebfcf690614dac0b77dc3c92c8cdca"}
Sep 10 15:42:07.808592 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:07.808561    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:42:07.838677 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:07.838615    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb" podStartSLOduration=8.175820547 podStartE2EDuration="22.83859381s" totalImagesPullingTime="14.662773263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:07.837647151 +0000 UTC m=+23.753772051" watchObservedRunningTime="2026-09-10 15:42:07.83859381 +0000 UTC m=+23.754718690"
Sep 10 15:42:08.557947 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:08.557898    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:08.558650 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:08.558060    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:42:08.796343 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:08.796309    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:42:08.796511 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:08.796411    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:42:08.816329 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:08.816236    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:42:09.224688 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:09.224604    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5c6r7"]
Sep 10 15:42:09.224865 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:09.224762    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:09.224926 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:09.224892    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:42:09.227570 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:09.227539    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qswkd"]
Sep 10 15:42:09.227721 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:09.227655    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:09.227782 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:09.227745    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:42:09.231881 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:09.231850    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9ct5"]
Sep 10 15:42:09.232028 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:09.231988    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:09.232125 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:09.232100    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:42:10.557640 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:10.557603    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:10.558461 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:10.557753    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:42:10.558461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:10.557604    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:10.558461 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:10.558287    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:42:11.558011 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:11.557972    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:11.558482 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:11.558076    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:42:12.558055 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:12.557793    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:12.558537 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:12.557854    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:12.558537 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:12.558195    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9ct5" podUID="f0d7f839-f1d5-4657-aa6c-5b19b248622b"
Sep 10 15:42:12.558537 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:12.558261    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5c6r7" podUID="511e3fe0-d13f-413c-9877-1e056560de23"
Sep 10 15:42:12.806605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:12.806568    2527 generic.go:356] "Generic (PLEG): container finished" podID="a40c2ed6-ebbb-4c90-919c-23172dca982e" containerID="74ff90c50b577125e2be81c1b8c250aa77df55af1ed9ef993269d52f707b609e" exitCode=0
Sep 10 15:42:12.806803 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:12.806651    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps5pl" event={"ID":"a40c2ed6-ebbb-4c90-919c-23172dca982e","Type":"ContainerDied","Data":"74ff90c50b577125e2be81c1b8c250aa77df55af1ed9ef993269d52f707b609e"}
Sep 10 15:42:12.808284 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:12.808224    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sv2hv" event={"ID":"3bf9a25e-613d-4f16-8e38-79be9d27bdff","Type":"ContainerStarted","Data":"3174f71babd3274e567ae9f79b0d94d634d80ae4aba3bcd1d78465efb12e8bfa"}
Sep 10 15:42:12.863313 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:12.863263    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sv2hv" podStartSLOduration=1.895868468 podStartE2EDuration="27.863245657s" totalImagesPullingTime="25.967377189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-10 15:42:12.862916559 +0000 UTC m=+28.779041451" watchObservedRunningTime="2026-09-10 15:42:12.863245657 +0000 UTC m=+28.779370536"
Sep 10 15:42:13.557259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.557219    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:13.557435 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:13.557347    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qswkd" podUID="332552b5-8c5f-4050-81f0-cf2d5377dd09"
Sep 10 15:42:13.880097 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.880065    2527 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-61.ec2.internal" event="NodeReady"
Sep 10 15:42:13.880663 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.880208    2527 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 15:42:13.917881 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.917657    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67ddd8b586-7pt28"]
Sep 10 15:42:13.928765 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.928725    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"]
Sep 10 15:42:13.928962 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.928874    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:13.934959 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.934921    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hck2z\""
Sep 10 15:42:13.935122 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.934968    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 15:42:13.935122 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.934975    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 15:42:13.935122 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.934921    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 15:42:13.943488 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.943455    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 15:42:13.950648 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.950619    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-595p9"]
Sep 10 15:42:13.950818 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.950771    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:13.953871 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.953677    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 15:42:13.953871 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.953692    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 15:42:13.953871 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.953751    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tc2md\""
Sep 10 15:42:13.953871 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.953756    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 15:42:13.953871 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.953763    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 15:42:13.971686 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.971645    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5cf6fc6776-qgwn4"]
Sep 10 15:42:13.971870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.971774    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:13.974067 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.974022    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 15:42:13.974067 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.974045    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 15:42:13.974379 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.974049    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:42:13.974379 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.974193    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mgw65\""
Sep 10 15:42:13.974379 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.974258    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 15:42:13.979442 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.979417    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 15:42:13.995894 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.995862    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-5gl8t"]
Sep 10 15:42:13.996055 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.996007    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:13.998018 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.997998    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 15:42:13.998237 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.998223    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 15:42:13.998334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.998288    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d968c\""
Sep 10 15:42:13.998334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.998227    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 15:42:13.998334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.998288    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 15:42:13.998478 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.998341    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 15:42:13.998478 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:13.998330    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 15:42:14.005636 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.005614    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj"]
Sep 10 15:42:14.005776 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.005760    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.007603 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007574    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.007719 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007635    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-image-registry-private-configuration\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.007719 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007671    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-installation-pull-secrets\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.007822 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007730    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sw7n\" (UniqueName: \"kubernetes.io/projected/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-kube-api-access-7sw7n\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:14.007822 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007757    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7q8v\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-kube-api-access-s7q8v\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.007822 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007812    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-bound-sa-token\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.007962 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007864    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-trusted-ca\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.007962 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007924    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:14.007962 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007953    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-certificates\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.008094 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.007987    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-ca-trust-extracted\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.008094 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.008030    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:14.008635 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.008610    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 15:42:14.008870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.008812    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 15:42:14.009034 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.008904    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 15:42:14.009034 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.008978    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xxjhr\""
Sep 10 15:42:14.009154 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.009114    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 15:42:14.014873 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.014849    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"]
Sep 10 15:42:14.014987 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.014976    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj"
Sep 10 15:42:14.015833 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.015813    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 15:42:14.021965 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.021944    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-lf8qj\""
Sep 10 15:42:14.022071 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.021946    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:42:14.022071 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.022012    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 15:42:14.026336 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.026316    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wnlqg"]
Sep 10 15:42:14.026450 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.026435    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.028843 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.028823    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 15:42:14.028958 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.028877    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 15:42:14.028958 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.028905    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 15:42:14.029041 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.028963    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hlrx8\""
Sep 10 15:42:14.029041 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.028910    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:42:14.035251 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.035230    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"]
Sep 10 15:42:14.035374 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.035336    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.037739 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.037720    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 15:42:14.037852 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.037725    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 15:42:14.037991 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.037978    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-brxch\""
Sep 10 15:42:14.059334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.059292    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"]
Sep 10 15:42:14.059535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.059436    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.061961 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.061935    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 15:42:14.061961 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.061946    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 15:42:14.062212 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.062021    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 15:42:14.062257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.062237    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 15:42:14.073611 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.073585    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"]
Sep 10 15:42:14.073755 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.073735    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.076392 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.076370    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 15:42:14.076597 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.076464    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 15:42:14.076694 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.076491    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-w6whb\""
Sep 10 15:42:14.076758 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.076495    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 15:42:14.076882 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.076865    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:42:14.095565 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.095484    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-h5mql"]
Sep 10 15:42:14.096101 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.096064    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:14.099232 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.099206    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6qzgp\""
Sep 10 15:42:14.099603 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.099576    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:42:14.099913 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.099894    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 10 15:42:14.100070 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.100050    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 10 15:42:14.108077 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108049    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-d47zd"]
Sep 10 15:42:14.108523 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108447    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h5mql"
Sep 10 15:42:14.108523 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108476    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-stats-auth\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.108523 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108513    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:14.108689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108540    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d30a715d-e7b9-4e9e-b311-2675c91df3df-serving-cert\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.108689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108572    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-image-registry-private-configuration\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.108689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108598    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8101fd3c-1dcc-4099-88af-6581442ee257-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.108689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108630    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6jdn\" (UniqueName: \"kubernetes.io/projected/c8fd6930-9ef8-4269-8e75-370c3f2d4deb-kube-api-access-w6jdn\") pod \"volume-data-source-validator-695c895bd6-58bxj\" (UID: \"c8fd6930-9ef8-4269-8e75-370c3f2d4deb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj"
Sep 10 15:42:14.108689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108654    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snxwb\" (UniqueName: \"kubernetes.io/projected/8101fd3c-1dcc-4099-88af-6581442ee257-kube-api-access-snxwb\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108693    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drnfs\" (UniqueName: \"kubernetes.io/projected/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-kube-api-access-drnfs\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108720    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-config-volume\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108746    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-default-certificate\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108769    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108792    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8101fd3c-1dcc-4099-88af-6581442ee257-tmp\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108812    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8101fd3c-1dcc-4099-88af-6581442ee257-snapshots\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108839    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sw7n\" (UniqueName: \"kubernetes.io/projected/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-kube-api-access-7sw7n\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108862    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-tmp-dir\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108894    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-trusted-ca\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.108932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108932    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee34d318-9263-4aeb-a95f-316447f94aa2-serving-cert\") pod \"service-ca-operator-859f94c4b-fjlvc\" (UID: \"ee34d318-9263-4aeb-a95f-316447f94aa2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108957    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8101fd3c-1dcc-4099-88af-6581442ee257-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.108995    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8101fd3c-1dcc-4099-88af-6581442ee257-serving-cert\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109021    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109044    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109129    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee34d318-9263-4aeb-a95f-316447f94aa2-config\") pod \"service-ca-operator-859f94c4b-fjlvc\" (UID: \"ee34d318-9263-4aeb-a95f-316447f94aa2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109202    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-554l4\" (UniqueName: \"kubernetes.io/projected/ee34d318-9263-4aeb-a95f-316447f94aa2-kube-api-access-554l4\") pod \"service-ca-operator-859f94c4b-fjlvc\" (UID: \"ee34d318-9263-4aeb-a95f-316447f94aa2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109250    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtwhj\" (UniqueName: \"kubernetes.io/projected/e6549d43-a327-43b8-b3a9-0c71532c4985-kube-api-access-qtwhj\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109307    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109332    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.109417 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.109414    2527 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:42:14.109861 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.109428    2527 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67ddd8b586-7pt28: secret "image-registry-tls" not found
Sep 10 15:42:14.109861 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.109479    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls podName:abc19d51-c8e1-4f57-88c8-2004ec54c4bb nodeName:}" failed. No retries permitted until 2026-09-10 15:42:14.609460136 +0000 UTC m=+30.525585004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls") pod "image-registry-67ddd8b586-7pt28" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb") : secret "image-registry-tls" not found
Sep 10 15:42:14.109861 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109701    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67t7w\" (UniqueName: \"kubernetes.io/projected/d30a715d-e7b9-4e9e-b311-2675c91df3df-kube-api-access-67t7w\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.109861 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109765    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-installation-pull-secrets\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.109861 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.109795    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7q8v\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-kube-api-access-s7q8v\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.110103 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.110031    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-trusted-ca\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.110270 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.110227    2527 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:14.110330 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.110295    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls podName:137e05f8-fbb8-45bb-af7e-418ecd5f1f51 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:14.610276193 +0000 UTC m=+30.526401065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-tggpc" (UID: "137e05f8-fbb8-45bb-af7e-418ecd5f1f51") : secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:14.110377 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.110325    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-bound-sa-token\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.110377 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.110359    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d30a715d-e7b9-4e9e-b311-2675c91df3df-config\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.110461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.110385    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d30a715d-e7b9-4e9e-b311-2675c91df3df-trusted-ca\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.110461 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.110416    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-certificates\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.110546 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.110456    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-ca-trust-extracted\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.110759 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.110735    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-ca-trust-extracted\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.111108 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.111088    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-certificates\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.111190 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.111104    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:14.114256 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.114224    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-installation-pull-secrets\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.114396 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.114229    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-image-registry-private-configuration\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.116045 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.116026    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 15:42:14.116298 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.116282    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 15:42:14.116385 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.116313    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pdj5g\""
Sep 10 15:42:14.116385 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.116362    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 15:42:14.124977 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.124939    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7q8v\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-kube-api-access-s7q8v\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.125110 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.124986    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sw7n\" (UniqueName: \"kubernetes.io/projected/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-kube-api-access-7sw7n\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:14.125110 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.125015    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-bound-sa-token\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.135060 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.135031    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-pxwpp"]
Sep 10 15:42:14.135226 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.135163    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:14.137391 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.137365    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 15:42:14.137505 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.137428    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 15:42:14.137552 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.137536    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-njtv5\""
Sep 10 15:42:14.164795 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.164769    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"]
Sep 10 15:42:14.164927 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.164889    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-pxwpp"
Sep 10 15:42:14.167393 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.167372    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 15:42:14.167584 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.167567    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-q4j9v\""
Sep 10 15:42:14.167676 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.167662    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 15:42:14.182944 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.182919    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"]
Sep 10 15:42:14.183098 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.183056    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.185537 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.185511    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 15:42:14.185659 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.185522    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 15:42:14.185659 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.185574    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 15:42:14.185659 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.185618    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 15:42:14.200436 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200408    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-595p9"]
Sep 10 15:42:14.200628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200458    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"]
Sep 10 15:42:14.200628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200482    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67ddd8b586-7pt28"]
Sep 10 15:42:14.200628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200502    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-5gl8t"]
Sep 10 15:42:14.200628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200520    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5cf6fc6776-qgwn4"]
Sep 10 15:42:14.200628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200538    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wnlqg"]
Sep 10 15:42:14.200628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200562    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h5mql"]
Sep 10 15:42:14.200628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200582    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"]
Sep 10 15:42:14.200628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200600    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-pxwpp"]
Sep 10 15:42:14.200628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200619    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"]
Sep 10 15:42:14.201049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200635    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj"]
Sep 10 15:42:14.201049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200655    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"]
Sep 10 15:42:14.201049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200562    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"
Sep 10 15:42:14.201049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200726    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-d47zd"]
Sep 10 15:42:14.201049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200749    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"]
Sep 10 15:42:14.201049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200932    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"]
Sep 10 15:42:14.201049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.200959    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"]
Sep 10 15:42:14.204329 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.204310    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 15:42:14.204453 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.204433    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jzrj4\""
Sep 10 15:42:14.211151 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211105    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b7b241e-708b-40bb-ba3d-52c0fee5e425-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7bdj4\" (UID: \"2b7b241e-708b-40bb-ba3d-52c0fee5e425\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.211256 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211162    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d30a715d-e7b9-4e9e-b311-2675c91df3df-config\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.211256 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211223    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d30a715d-e7b9-4e9e-b311-2675c91df3df-trusted-ca\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.211342 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211273    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qzrt\" (UniqueName: \"kubernetes.io/projected/83488306-a407-4c7f-9309-d7d47955d038-kube-api-access-9qzrt\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:14.211342 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211305    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b7b241e-708b-40bb-ba3d-52c0fee5e425-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7bdj4\" (UID: \"2b7b241e-708b-40bb-ba3d-52c0fee5e425\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.211342 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211333    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-stats-auth\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.211484 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211364    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d30a715d-e7b9-4e9e-b311-2675c91df3df-serving-cert\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.211484 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211419    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8101fd3c-1dcc-4099-88af-6581442ee257-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.211484 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211465    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6jdn\" (UniqueName: \"kubernetes.io/projected/c8fd6930-9ef8-4269-8e75-370c3f2d4deb-kube-api-access-w6jdn\") pod \"volume-data-source-validator-695c895bd6-58bxj\" (UID: \"c8fd6930-9ef8-4269-8e75-370c3f2d4deb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj"
Sep 10 15:42:14.211631 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211495    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snxwb\" (UniqueName: \"kubernetes.io/projected/8101fd3c-1dcc-4099-88af-6581442ee257-kube-api-access-snxwb\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.211631 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211541    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drnfs\" (UniqueName: \"kubernetes.io/projected/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-kube-api-access-drnfs\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.211631 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211569    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:14.211631 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211597    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-config-volume\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.211631 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211622    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-default-certificate\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.211870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211646    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.211870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211671    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8101fd3c-1dcc-4099-88af-6581442ee257-tmp\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.211870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211695    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17f2efec-4d06-45ed-8f25-4c95181e821e-tmp\") pod \"klusterlet-addon-workmgr-7fb8f8b574-d7mw6\" (UID: \"17f2efec-4d06-45ed-8f25-4c95181e821e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.211870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211717    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8101fd3c-1dcc-4099-88af-6581442ee257-snapshots\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.211870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211742    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-tmp-dir\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.211870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211767    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2mpj\" (UniqueName: \"kubernetes.io/projected/2120541f-3d27-4e5e-8439-374a8d237e26-kube-api-access-t2mpj\") pod \"ingress-canary-h5mql\" (UID: \"2120541f-3d27-4e5e-8439-374a8d237e26\") " pod="openshift-ingress-canary/ingress-canary-h5mql"
Sep 10 15:42:14.211870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211816    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee34d318-9263-4aeb-a95f-316447f94aa2-serving-cert\") pod \"service-ca-operator-859f94c4b-fjlvc\" (UID: \"ee34d318-9263-4aeb-a95f-316447f94aa2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.211870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211846    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8101fd3c-1dcc-4099-88af-6581442ee257-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.211870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211869    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8101fd3c-1dcc-4099-88af-6581442ee257-serving-cert\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211904    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211930    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee34d318-9263-4aeb-a95f-316447f94aa2-config\") pod \"service-ca-operator-859f94c4b-fjlvc\" (UID: \"ee34d318-9263-4aeb-a95f-316447f94aa2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211959    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.211989    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgrrp\" (UniqueName: \"kubernetes.io/projected/2b7b241e-708b-40bb-ba3d-52c0fee5e425-kube-api-access-kgrrp\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7bdj4\" (UID: \"2b7b241e-708b-40bb-ba3d-52c0fee5e425\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212021    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-554l4\" (UniqueName: \"kubernetes.io/projected/ee34d318-9263-4aeb-a95f-316447f94aa2-kube-api-access-554l4\") pod \"service-ca-operator-859f94c4b-fjlvc\" (UID: \"ee34d318-9263-4aeb-a95f-316447f94aa2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212027    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d30a715d-e7b9-4e9e-b311-2675c91df3df-config\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212055    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtwhj\" (UniqueName: \"kubernetes.io/projected/e6549d43-a327-43b8-b3a9-0c71532c4985-kube-api-access-qtwhj\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212095    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d30a715d-e7b9-4e9e-b311-2675c91df3df-trusted-ca\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212118    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212169    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/078c1103-50b4-4a77-ba94-0d1e164c5b41-nginx-conf\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212276    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2120541f-3d27-4e5e-8439-374a8d237e26-cert\") pod \"ingress-canary-h5mql\" (UID: \"2120541f-3d27-4e5e-8439-374a8d237e26\") " pod="openshift-ingress-canary/ingress-canary-h5mql"
Sep 10 15:42:14.212314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212316    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67t7w\" (UniqueName: \"kubernetes.io/projected/d30a715d-e7b9-4e9e-b311-2675c91df3df-kube-api-access-67t7w\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.212852 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212352    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/17f2efec-4d06-45ed-8f25-4c95181e821e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7fb8f8b574-d7mw6\" (UID: \"17f2efec-4d06-45ed-8f25-4c95181e821e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.212852 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212391    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d42m\" (UniqueName: \"kubernetes.io/projected/17f2efec-4d06-45ed-8f25-4c95181e821e-kube-api-access-7d42m\") pod \"klusterlet-addon-workmgr-7fb8f8b574-d7mw6\" (UID: \"17f2efec-4d06-45ed-8f25-4c95181e821e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.212852 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.212400    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-config-volume\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.213048 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.213027    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:14.713006807 +0000 UTC m=+30.629131665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : configmap references non-existent config key: service-ca.crt
Sep 10 15:42:14.214032 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.213490    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8101fd3c-1dcc-4099-88af-6581442ee257-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.214032 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.213520    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8101fd3c-1dcc-4099-88af-6581442ee257-tmp\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.214032 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.213610    2527 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:42:14.214032 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.213657    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls podName:909c47d6-65aa-4f4f-bfc3-da05c15bc3c9 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:14.713642229 +0000 UTC m=+30.629767087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls") pod "dns-default-wnlqg" (UID: "909c47d6-65aa-4f4f-bfc3-da05c15bc3c9") : secret "dns-default-metrics-tls" not found
Sep 10 15:42:14.214346 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.214106    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-tmp-dir\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.214687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.214667    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8101fd3c-1dcc-4099-88af-6581442ee257-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.214801 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.214692    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8101fd3c-1dcc-4099-88af-6581442ee257-snapshots\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.214889 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.214772    2527 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 15:42:14.215024 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.215004    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:14.714984917 +0000 UTC m=+30.631109793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : secret "router-metrics-certs-default" not found
Sep 10 15:42:14.215425 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.215067    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-default-certificate\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.215425 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.215288    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee34d318-9263-4aeb-a95f-316447f94aa2-config\") pod \"service-ca-operator-859f94c4b-fjlvc\" (UID: \"ee34d318-9263-4aeb-a95f-316447f94aa2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.217686 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.217455    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-stats-auth\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.217894 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.217770    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8101fd3c-1dcc-4099-88af-6581442ee257-serving-cert\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.218182 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.218118    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d30a715d-e7b9-4e9e-b311-2675c91df3df-serving-cert\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.218361 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.218343    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee34d318-9263-4aeb-a95f-316447f94aa2-serving-cert\") pod \"service-ca-operator-859f94c4b-fjlvc\" (UID: \"ee34d318-9263-4aeb-a95f-316447f94aa2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.222712 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.222676    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6jdn\" (UniqueName: \"kubernetes.io/projected/c8fd6930-9ef8-4269-8e75-370c3f2d4deb-kube-api-access-w6jdn\") pod \"volume-data-source-validator-695c895bd6-58bxj\" (UID: \"c8fd6930-9ef8-4269-8e75-370c3f2d4deb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj"
Sep 10 15:42:14.223847 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.223822    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drnfs\" (UniqueName: \"kubernetes.io/projected/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-kube-api-access-drnfs\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.224260 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.224236    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-554l4\" (UniqueName: \"kubernetes.io/projected/ee34d318-9263-4aeb-a95f-316447f94aa2-kube-api-access-554l4\") pod \"service-ca-operator-859f94c4b-fjlvc\" (UID: \"ee34d318-9263-4aeb-a95f-316447f94aa2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.224370 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.224313    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtwhj\" (UniqueName: \"kubernetes.io/projected/e6549d43-a327-43b8-b3a9-0c71532c4985-kube-api-access-qtwhj\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.224520 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.224499    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snxwb\" (UniqueName: \"kubernetes.io/projected/8101fd3c-1dcc-4099-88af-6581442ee257-kube-api-access-snxwb\") pod \"insights-operator-76bcd6ccf5-5gl8t\" (UID: \"8101fd3c-1dcc-4099-88af-6581442ee257\") " pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.225248 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.225227    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67t7w\" (UniqueName: \"kubernetes.io/projected/d30a715d-e7b9-4e9e-b311-2675c91df3df-kube-api-access-67t7w\") pod \"console-operator-85b454874b-595p9\" (UID: \"d30a715d-e7b9-4e9e-b311-2675c91df3df\") " pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.282706 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.282659    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:14.313150 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313096    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/078c1103-50b4-4a77-ba94-0d1e164c5b41-nginx-conf\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:14.313150 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313151    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2120541f-3d27-4e5e-8439-374a8d237e26-cert\") pod \"ingress-canary-h5mql\" (UID: \"2120541f-3d27-4e5e-8439-374a8d237e26\") " pod="openshift-ingress-canary/ingress-canary-h5mql"
Sep 10 15:42:14.313286 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313180    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e839405-30b3-4105-80d1-3d29a1fe8683-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-586bb8f644-wx5wx\" (UID: \"3e839405-30b3-4105-80d1-3d29a1fe8683\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"
Sep 10 15:42:14.313286 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313210    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/17f2efec-4d06-45ed-8f25-4c95181e821e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7fb8f8b574-d7mw6\" (UID: \"17f2efec-4d06-45ed-8f25-4c95181e821e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.313286 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313244    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d42m\" (UniqueName: \"kubernetes.io/projected/17f2efec-4d06-45ed-8f25-4c95181e821e-kube-api-access-7d42m\") pod \"klusterlet-addon-workmgr-7fb8f8b574-d7mw6\" (UID: \"17f2efec-4d06-45ed-8f25-4c95181e821e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.313286 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313267    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b7b241e-708b-40bb-ba3d-52c0fee5e425-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7bdj4\" (UID: \"2b7b241e-708b-40bb-ba3d-52c0fee5e425\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.313444 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313290    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tw8k\" (UniqueName: \"kubernetes.io/projected/ae5f449d-8e49-47ca-976a-b63650f74b6a-kube-api-access-4tw8k\") pod \"network-check-source-685456655f-pxwpp\" (UID: \"ae5f449d-8e49-47ca-976a-b63650f74b6a\") " pod="openshift-network-diagnostics/network-check-source-685456655f-pxwpp"
Sep 10 15:42:14.313444 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313347    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-hub\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.313444 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313390    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.313444 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313430    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qzrt\" (UniqueName: \"kubernetes.io/projected/83488306-a407-4c7f-9309-d7d47955d038-kube-api-access-9qzrt\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:14.313605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313460    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b7b241e-708b-40bb-ba3d-52c0fee5e425-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7bdj4\" (UID: \"2b7b241e-708b-40bb-ba3d-52c0fee5e425\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.313605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313486    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.313605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313511    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f934b5-d4b9-49af-a439-0e77d6890343-additional-service-ca\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.313822 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313798    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:14.313887 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313850    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17f2efec-4d06-45ed-8f25-4c95181e821e-tmp\") pod \"klusterlet-addon-workmgr-7fb8f8b574-d7mw6\" (UID: \"17f2efec-4d06-45ed-8f25-4c95181e821e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.313887 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313859    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b7b241e-708b-40bb-ba3d-52c0fee5e425-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7bdj4\" (UID: \"2b7b241e-708b-40bb-ba3d-52c0fee5e425\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.313887 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313881    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2mpj\" (UniqueName: \"kubernetes.io/projected/2120541f-3d27-4e5e-8439-374a8d237e26-kube-api-access-t2mpj\") pod \"ingress-canary-h5mql\" (UID: \"2120541f-3d27-4e5e-8439-374a8d237e26\") " pod="openshift-ingress-canary/ingress-canary-h5mql"
Sep 10 15:42:14.314026 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313915    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/078c1103-50b4-4a77-ba94-0d1e164c5b41-nginx-conf\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:14.314026 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.313884    2527 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 15:42:14.314026 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313935    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vw9n\" (UniqueName: \"kubernetes.io/projected/3e839405-30b3-4105-80d1-3d29a1fe8683-kube-api-access-5vw9n\") pod \"managed-serviceaccount-addon-agent-586bb8f644-wx5wx\" (UID: \"3e839405-30b3-4105-80d1-3d29a1fe8683\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"
Sep 10 15:42:14.314026 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.313963    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-ca\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.314026 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.313991    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert podName:078c1103-50b4-4a77-ba94-0d1e164c5b41 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:14.813973973 +0000 UTC m=+30.730098845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d47zd" (UID: "078c1103-50b4-4a77-ba94-0d1e164c5b41") : secret "networking-console-plugin-cert" not found
Sep 10 15:42:14.314313 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.314077    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr6wb\" (UniqueName: \"kubernetes.io/projected/67f934b5-d4b9-49af-a439-0e77d6890343-kube-api-access-rr6wb\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.314313 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.314112    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:14.314313 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.314162    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgrrp\" (UniqueName: \"kubernetes.io/projected/2b7b241e-708b-40bb-ba3d-52c0fee5e425-kube-api-access-kgrrp\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7bdj4\" (UID: \"2b7b241e-708b-40bb-ba3d-52c0fee5e425\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.314313 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.314195    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17f2efec-4d06-45ed-8f25-4c95181e821e-tmp\") pod \"klusterlet-addon-workmgr-7fb8f8b574-d7mw6\" (UID: \"17f2efec-4d06-45ed-8f25-4c95181e821e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.314313 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.314235    2527 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 15:42:14.314313 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.314292    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls podName:83488306-a407-4c7f-9309-d7d47955d038 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:14.814277843 +0000 UTC m=+30.730402710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qw66j" (UID: "83488306-a407-4c7f-9309-d7d47955d038") : secret "samples-operator-tls" not found
Sep 10 15:42:14.315858 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.315836    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2120541f-3d27-4e5e-8439-374a8d237e26-cert\") pod \"ingress-canary-h5mql\" (UID: \"2120541f-3d27-4e5e-8439-374a8d237e26\") " pod="openshift-ingress-canary/ingress-canary-h5mql"
Sep 10 15:42:14.316051 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.316034    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b7b241e-708b-40bb-ba3d-52c0fee5e425-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7bdj4\" (UID: \"2b7b241e-708b-40bb-ba3d-52c0fee5e425\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.316318 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.316303    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/17f2efec-4d06-45ed-8f25-4c95181e821e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7fb8f8b574-d7mw6\" (UID: \"17f2efec-4d06-45ed-8f25-4c95181e821e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.317020 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.317008    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t"
Sep 10 15:42:14.323435 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.323267    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d42m\" (UniqueName: \"kubernetes.io/projected/17f2efec-4d06-45ed-8f25-4c95181e821e-kube-api-access-7d42m\") pod \"klusterlet-addon-workmgr-7fb8f8b574-d7mw6\" (UID: \"17f2efec-4d06-45ed-8f25-4c95181e821e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.323567 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.323279    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2mpj\" (UniqueName: \"kubernetes.io/projected/2120541f-3d27-4e5e-8439-374a8d237e26-kube-api-access-t2mpj\") pod \"ingress-canary-h5mql\" (UID: \"2120541f-3d27-4e5e-8439-374a8d237e26\") " pod="openshift-ingress-canary/ingress-canary-h5mql"
Sep 10 15:42:14.323567 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.323553    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgrrp\" (UniqueName: \"kubernetes.io/projected/2b7b241e-708b-40bb-ba3d-52c0fee5e425-kube-api-access-kgrrp\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7bdj4\" (UID: \"2b7b241e-708b-40bb-ba3d-52c0fee5e425\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.323683 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.323637    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj"
Sep 10 15:42:14.323758 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.323740    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qzrt\" (UniqueName: \"kubernetes.io/projected/83488306-a407-4c7f-9309-d7d47955d038-kube-api-access-9qzrt\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:14.335564 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.335533    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"
Sep 10 15:42:14.370072 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.370050    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:14.389319 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.389269    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"
Sep 10 15:42:14.416250 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.415897    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tw8k\" (UniqueName: \"kubernetes.io/projected/ae5f449d-8e49-47ca-976a-b63650f74b6a-kube-api-access-4tw8k\") pod \"network-check-source-685456655f-pxwpp\" (UID: \"ae5f449d-8e49-47ca-976a-b63650f74b6a\") " pod="openshift-network-diagnostics/network-check-source-685456655f-pxwpp"
Sep 10 15:42:14.416250 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.415939    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-hub\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.416250 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.415966    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.418953 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.418150    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.418953 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.418224    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f934b5-d4b9-49af-a439-0e77d6890343-additional-service-ca\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.418953 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.418369    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vw9n\" (UniqueName: \"kubernetes.io/projected/3e839405-30b3-4105-80d1-3d29a1fe8683-kube-api-access-5vw9n\") pod \"managed-serviceaccount-addon-agent-586bb8f644-wx5wx\" (UID: \"3e839405-30b3-4105-80d1-3d29a1fe8683\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"
Sep 10 15:42:14.418953 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.418394    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-ca\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.418953 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.418441    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr6wb\" (UniqueName: \"kubernetes.io/projected/67f934b5-d4b9-49af-a439-0e77d6890343-kube-api-access-rr6wb\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.418953 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.418539    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e839405-30b3-4105-80d1-3d29a1fe8683-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-586bb8f644-wx5wx\" (UID: \"3e839405-30b3-4105-80d1-3d29a1fe8683\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"
Sep 10 15:42:14.419362 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.419064    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f934b5-d4b9-49af-a439-0e77d6890343-additional-service-ca\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.420008 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.419963    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-hub\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.422178 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.422062    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.424366 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.424080    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-ca\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.424366 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.424264    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/67f934b5-d4b9-49af-a439-0e77d6890343-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.425149 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.425108    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e839405-30b3-4105-80d1-3d29a1fe8683-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-586bb8f644-wx5wx\" (UID: \"3e839405-30b3-4105-80d1-3d29a1fe8683\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"
Sep 10 15:42:14.428819 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.428758    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vw9n\" (UniqueName: \"kubernetes.io/projected/3e839405-30b3-4105-80d1-3d29a1fe8683-kube-api-access-5vw9n\") pod \"managed-serviceaccount-addon-agent-586bb8f644-wx5wx\" (UID: \"3e839405-30b3-4105-80d1-3d29a1fe8683\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"
Sep 10 15:42:14.428819 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.428802    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tw8k\" (UniqueName: \"kubernetes.io/projected/ae5f449d-8e49-47ca-976a-b63650f74b6a-kube-api-access-4tw8k\") pod \"network-check-source-685456655f-pxwpp\" (UID: \"ae5f449d-8e49-47ca-976a-b63650f74b6a\") " pod="openshift-network-diagnostics/network-check-source-685456655f-pxwpp"
Sep 10 15:42:14.431284 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.431098    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr6wb\" (UniqueName: \"kubernetes.io/projected/67f934b5-d4b9-49af-a439-0e77d6890343-kube-api-access-rr6wb\") pod \"cluster-proxy-proxy-agent-868d69f696-nmb5q\" (UID: \"67f934b5-d4b9-49af-a439-0e77d6890343\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.449738 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.449710    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h5mql"
Sep 10 15:42:14.474429 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.474407    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-pxwpp"
Sep 10 15:42:14.502791 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.501989    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:42:14.514555 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.512404    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"
Sep 10 15:42:14.560300 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.559647    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:14.560300 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.559849    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:14.570118 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.565199    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 15:42:14.570118 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.565270    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 15:42:14.570118 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.566874    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2gdj6\""
Sep 10 15:42:14.621248 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.621091    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:14.621415 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.621332    2527 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:14.621490 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.621434    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls podName:137e05f8-fbb8-45bb-af7e-418ecd5f1f51 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:15.621408793 +0000 UTC m=+31.537533651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-tggpc" (UID: "137e05f8-fbb8-45bb-af7e-418ecd5f1f51") : secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:14.621490 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.621463    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:14.621623 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.621607    2527 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:42:14.621623 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.621624    2527 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67ddd8b586-7pt28: secret "image-registry-tls" not found
Sep 10 15:42:14.621701 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.621660    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls podName:abc19d51-c8e1-4f57-88c8-2004ec54c4bb nodeName:}" failed. No retries permitted until 2026-09-10 15:42:15.621651326 +0000 UTC m=+31.537776184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls") pod "image-registry-67ddd8b586-7pt28" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb") : secret "image-registry-tls" not found
Sep 10 15:42:14.630041 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.630018    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-5gl8t"]
Sep 10 15:42:14.632783 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.632741    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-595p9"]
Sep 10 15:42:14.634308 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.634277    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8101fd3c_1dcc_4099_88af_6581442ee257.slice/crio-4891155598a850be8fb2801c0926f9f044669bb42fdbd8e74d6c81c25b8e7b42 WatchSource:0}: Error finding container 4891155598a850be8fb2801c0926f9f044669bb42fdbd8e74d6c81c25b8e7b42: Status 404 returned error can't find the container with id 4891155598a850be8fb2801c0926f9f044669bb42fdbd8e74d6c81c25b8e7b42
Sep 10 15:42:14.635644 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.635617    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd30a715d_e7b9_4e9e_b311_2675c91df3df.slice/crio-7a6a4715768e09001808df3244585f095fb7f30f8710119ba50174b9df7f660e WatchSource:0}: Error finding container 7a6a4715768e09001808df3244585f095fb7f30f8710119ba50174b9df7f660e: Status 404 returned error can't find the container with id 7a6a4715768e09001808df3244585f095fb7f30f8710119ba50174b9df7f660e
Sep 10 15:42:14.652711 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.652662    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj"]
Sep 10 15:42:14.655770 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.655738    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8fd6930_9ef8_4269_8e75_370c3f2d4deb.slice/crio-524d93ca5d1c5ab23237ccc09f009499466ac8667a7f196fe2f5b6d48389eed0 WatchSource:0}: Error finding container 524d93ca5d1c5ab23237ccc09f009499466ac8667a7f196fe2f5b6d48389eed0: Status 404 returned error can't find the container with id 524d93ca5d1c5ab23237ccc09f009499466ac8667a7f196fe2f5b6d48389eed0
Sep 10 15:42:14.696498 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.696411    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc"]
Sep 10 15:42:14.699364 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.699334    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee34d318_9263_4aeb_a95f_316447f94aa2.slice/crio-1e2ec8e47c5a3ff7bfcd3fce210e800e1c3b0da6981584807f286d62cf09c448 WatchSource:0}: Error finding container 1e2ec8e47c5a3ff7bfcd3fce210e800e1c3b0da6981584807f286d62cf09c448: Status 404 returned error can't find the container with id 1e2ec8e47c5a3ff7bfcd3fce210e800e1c3b0da6981584807f286d62cf09c448
Sep 10 15:42:14.712601 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.712571    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"]
Sep 10 15:42:14.715571 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.715537    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f2efec_4d06_45ed_8f25_4c95181e821e.slice/crio-ca19bcd24deb0158845f03cb0280891c029bc5f6c30da40285f6d52b72808749 WatchSource:0}: Error finding container ca19bcd24deb0158845f03cb0280891c029bc5f6c30da40285f6d52b72808749: Status 404 returned error can't find the container with id ca19bcd24deb0158845f03cb0280891c029bc5f6c30da40285f6d52b72808749
Sep 10 15:42:14.722822 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.722794    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.722937 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.722854    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:14.723052 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.723031    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:15.723006108 +0000 UTC m=+31.639130968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : configmap references non-existent config key: service-ca.crt
Sep 10 15:42:14.723184 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.723108    2527 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 15:42:14.723252 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.723207    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:15.723190941 +0000 UTC m=+31.639315798 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : secret "router-metrics-certs-default" not found
Sep 10 15:42:14.723252 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.723217    2527 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:42:14.723349 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.723269    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls podName:909c47d6-65aa-4f4f-bfc3-da05c15bc3c9 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:15.723256559 +0000 UTC m=+31.639381430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls") pod "dns-default-wnlqg" (UID: "909c47d6-65aa-4f4f-bfc3-da05c15bc3c9") : secret "dns-default-metrics-tls" not found
Sep 10 15:42:14.723349 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.723142    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:14.725281 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.725257    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4"]
Sep 10 15:42:14.727940 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.727916    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b7b241e_708b_40bb_ba3d_52c0fee5e425.slice/crio-393de1a591229effb349c156d95ef8f5213c30c58d1475fc75934caa5423df20 WatchSource:0}: Error finding container 393de1a591229effb349c156d95ef8f5213c30c58d1475fc75934caa5423df20: Status 404 returned error can't find the container with id 393de1a591229effb349c156d95ef8f5213c30c58d1475fc75934caa5423df20
Sep 10 15:42:14.772649 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.772590    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h5mql"]
Sep 10 15:42:14.775316 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.775289    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2120541f_3d27_4e5e_8439_374a8d237e26.slice/crio-2b4a1c94a6fea7327f59136780eff0a0292d607ce3b33d69f8f96fcb476ec9cf WatchSource:0}: Error finding container 2b4a1c94a6fea7327f59136780eff0a0292d607ce3b33d69f8f96fcb476ec9cf: Status 404 returned error can't find the container with id 2b4a1c94a6fea7327f59136780eff0a0292d607ce3b33d69f8f96fcb476ec9cf
Sep 10 15:42:14.807552 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.807519    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-pxwpp"]
Sep 10 15:42:14.810641 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.810591    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5f449d_8e49_47ca_976a_b63650f74b6a.slice/crio-31251c580566bd729c573f015043b10c404ce58ecea2b5cfebe426e6529d50a7 WatchSource:0}: Error finding container 31251c580566bd729c573f015043b10c404ce58ecea2b5cfebe426e6529d50a7: Status 404 returned error can't find the container with id 31251c580566bd729c573f015043b10c404ce58ecea2b5cfebe426e6529d50a7
Sep 10 15:42:14.813588 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.813562    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc" event={"ID":"ee34d318-9263-4aeb-a95f-316447f94aa2","Type":"ContainerStarted","Data":"1e2ec8e47c5a3ff7bfcd3fce210e800e1c3b0da6981584807f286d62cf09c448"}
Sep 10 15:42:14.815167 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.815111    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t" event={"ID":"8101fd3c-1dcc-4099-88af-6581442ee257","Type":"ContainerStarted","Data":"4891155598a850be8fb2801c0926f9f044669bb42fdbd8e74d6c81c25b8e7b42"}
Sep 10 15:42:14.816963 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.816901    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h5mql" event={"ID":"2120541f-3d27-4e5e-8439-374a8d237e26","Type":"ContainerStarted","Data":"2b4a1c94a6fea7327f59136780eff0a0292d607ce3b33d69f8f96fcb476ec9cf"}
Sep 10 15:42:14.818699 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.818670    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj" event={"ID":"c8fd6930-9ef8-4269-8e75-370c3f2d4deb","Type":"ContainerStarted","Data":"524d93ca5d1c5ab23237ccc09f009499466ac8667a7f196fe2f5b6d48389eed0"}
Sep 10 15:42:14.820711 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.820688    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"]
Sep 10 15:42:14.820861 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.820824    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4" event={"ID":"2b7b241e-708b-40bb-ba3d-52c0fee5e425","Type":"ContainerStarted","Data":"393de1a591229effb349c156d95ef8f5213c30c58d1475fc75934caa5423df20"}
Sep 10 15:42:14.822146 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.822108    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6" event={"ID":"17f2efec-4d06-45ed-8f25-4c95181e821e","Type":"ContainerStarted","Data":"ca19bcd24deb0158845f03cb0280891c029bc5f6c30da40285f6d52b72808749"}
Sep 10 15:42:14.823344 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.823318    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-595p9" event={"ID":"d30a715d-e7b9-4e9e-b311-2675c91df3df","Type":"ContainerStarted","Data":"7a6a4715768e09001808df3244585f095fb7f30f8710119ba50174b9df7f660e"}
Sep 10 15:42:14.823816 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.823793    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:14.823890 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.823868    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:14.823965 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.823951    2527 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 15:42:14.823965 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.823961    2527 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 15:42:14.824053 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.824022    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert podName:078c1103-50b4-4a77-ba94-0d1e164c5b41 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:15.824000068 +0000 UTC m=+31.740124935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d47zd" (UID: "078c1103-50b4-4a77-ba94-0d1e164c5b41") : secret "networking-console-plugin-cert" not found
Sep 10 15:42:14.824053 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.824041    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls podName:83488306-a407-4c7f-9309-d7d47955d038 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:15.824032107 +0000 UTC m=+31.740156966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qw66j" (UID: "83488306-a407-4c7f-9309-d7d47955d038") : secret "samples-operator-tls" not found
Sep 10 15:42:14.825454 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.825430    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67f934b5_d4b9_49af_a439_0e77d6890343.slice/crio-14a35db8d8d1eb11ca5f48bb0898244bde26f864eb2fc577d6adb52edbfdd95c WatchSource:0}: Error finding container 14a35db8d8d1eb11ca5f48bb0898244bde26f864eb2fc577d6adb52edbfdd95c: Status 404 returned error can't find the container with id 14a35db8d8d1eb11ca5f48bb0898244bde26f864eb2fc577d6adb52edbfdd95c
Sep 10 15:42:14.826156 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.826102    2527 generic.go:356] "Generic (PLEG): container finished" podID="a40c2ed6-ebbb-4c90-919c-23172dca982e" containerID="bf1e0de689c1fb28677f9f6e50ce323c4461ed0725da10fc985d3cd9baa268b5" exitCode=0
Sep 10 15:42:14.826218 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.826156    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps5pl" event={"ID":"a40c2ed6-ebbb-4c90-919c-23172dca982e","Type":"ContainerDied","Data":"bf1e0de689c1fb28677f9f6e50ce323c4461ed0725da10fc985d3cd9baa268b5"}
Sep 10 15:42:14.828191 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:14.828168    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx"]
Sep 10 15:42:14.831297 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:14.831277    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e839405_30b3_4105_80d1_3d29a1fe8683.slice/crio-aae9b73cdbcaceec12baf977db333f2b72292721b580068d0fd2e9a2b7f7504e WatchSource:0}: Error finding container aae9b73cdbcaceec12baf977db333f2b72292721b580068d0fd2e9a2b7f7504e: Status 404 returned error can't find the container with id aae9b73cdbcaceec12baf977db333f2b72292721b580068d0fd2e9a2b7f7504e
Sep 10 15:42:14.833568 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.833547    2527 kuberuntime_manager.go:1459] "Unhandled Error" err="container addon-agent start failed in pod managed-serviceaccount-addon-agent-586bb8f644-wx5wx_open-cluster-management-agent-addon(3e839405-30b3-4105-80d1-3d29a1fe8683): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 10 15:42:14.833630 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:14.833576    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ErrImagePull: \"pull QPS exceeded\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx" podUID="3e839405-30b3-4105-80d1-3d29a1fe8683"
Sep 10 15:42:15.557646 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.557590    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:15.560790 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.560554    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sjjtv\""
Sep 10 15:42:15.631897 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.631809    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:15.632075 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.632003    2527 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:15.632177 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.632083    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls podName:137e05f8-fbb8-45bb-af7e-418ecd5f1f51 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.632061543 +0000 UTC m=+33.548186417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-tggpc" (UID: "137e05f8-fbb8-45bb-af7e-418ecd5f1f51") : secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:15.632552 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.632301    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:15.632552 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.632546    2527 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:42:15.632700 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.632562    2527 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67ddd8b586-7pt28: secret "image-registry-tls" not found
Sep 10 15:42:15.632700 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.632618    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls podName:abc19d51-c8e1-4f57-88c8-2004ec54c4bb nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.632600874 +0000 UTC m=+33.548725734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls") pod "image-registry-67ddd8b586-7pt28" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb") : secret "image-registry-tls" not found
Sep 10 15:42:15.733771 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.733725    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:15.733951 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.733814    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:15.733951 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.733890    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:15.734264 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.734066    2527 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:42:15.734264 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.734162    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls podName:909c47d6-65aa-4f4f-bfc3-da05c15bc3c9 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.73412785 +0000 UTC m=+33.650252713 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls") pod "dns-default-wnlqg" (UID: "909c47d6-65aa-4f4f-bfc3-da05c15bc3c9") : secret "dns-default-metrics-tls" not found
Sep 10 15:42:15.734264 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.734199    2527 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 15:42:15.734264 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.734224    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.734205 +0000 UTC m=+33.650329874 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : configmap references non-existent config key: service-ca.crt
Sep 10 15:42:15.734264 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.734253    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.734237882 +0000 UTC m=+33.650362753 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : secret "router-metrics-certs-default" not found
Sep 10 15:42:15.833907 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.833777    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" event={"ID":"67f934b5-d4b9-49af-a439-0e77d6890343","Type":"ContainerStarted","Data":"14a35db8d8d1eb11ca5f48bb0898244bde26f864eb2fc577d6adb52edbfdd95c"}
Sep 10 15:42:15.835380 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.834563    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:15.835380 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.834662    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:15.835380 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.834934    2527 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 15:42:15.835380 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.834993    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls podName:83488306-a407-4c7f-9309-d7d47955d038 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.834974686 +0000 UTC m=+33.751099547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qw66j" (UID: "83488306-a407-4c7f-9309-d7d47955d038") : secret "samples-operator-tls" not found
Sep 10 15:42:15.835380 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.835244    2527 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 15:42:15.835380 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.835296    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert podName:078c1103-50b4-4a77-ba94-0d1e164c5b41 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:17.83527986 +0000 UTC m=+33.751404731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d47zd" (UID: "078c1103-50b4-4a77-ba94-0d1e164c5b41") : secret "networking-console-plugin-cert" not found
Sep 10 15:42:15.838281 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.837403    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx" event={"ID":"3e839405-30b3-4105-80d1-3d29a1fe8683","Type":"ContainerStarted","Data":"aae9b73cdbcaceec12baf977db333f2b72292721b580068d0fd2e9a2b7f7504e"}
Sep 10 15:42:15.841213 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:15.840991    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:e5c06c361ea9a17d00c0bc604d962ac51634baa422fc4c2fb181cf7c8cb495c7\\\": ErrImagePull: pull QPS exceeded\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx" podUID="3e839405-30b3-4105-80d1-3d29a1fe8683"
Sep 10 15:42:15.845733 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:15.845628    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-pxwpp" event={"ID":"ae5f449d-8e49-47ca-976a-b63650f74b6a","Type":"ContainerStarted","Data":"31251c580566bd729c573f015043b10c404ce58ecea2b5cfebe426e6529d50a7"}
Sep 10 15:42:16.860319 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:16.858892    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel9@sha256:e5c06c361ea9a17d00c0bc604d962ac51634baa422fc4c2fb181cf7c8cb495c7\\\": ErrImagePull: pull QPS exceeded\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx" podUID="3e839405-30b3-4105-80d1-3d29a1fe8683"
Sep 10 15:42:17.167330 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.167276    2527 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-v96j6_ca288a4f-6d91-475a-bdd0-22fb9c6c8361/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:17.266975 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.266898    2527 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-v96j6_ca288a4f-6d91-475a-bdd0-22fb9c6c8361/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:17.354454 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.353541    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:17.354454 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.353672    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:17.354454 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.353824    2527 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 15:42:17.354454 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.353881    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs podName:511e3fe0-d13f-413c-9877-1e056560de23 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:49.353861715 +0000 UTC m=+65.269986578 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs") pod "network-metrics-daemon-5c6r7" (UID: "511e3fe0-d13f-413c-9877-1e056560de23") : secret "metrics-daemon-secret" not found
Sep 10 15:42:17.365792 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.361219    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0d7f839-f1d5-4657-aa6c-5b19b248622b-original-pull-secret\") pod \"global-pull-secret-syncer-z9ct5\" (UID: \"f0d7f839-f1d5-4657-aa6c-5b19b248622b\") " pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:17.367263 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.367157    2527 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-v96j6_ca288a4f-6d91-475a-bdd0-22fb9c6c8361/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:17.455276 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.454493    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5pt\" (UniqueName: \"kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt\") pod \"network-check-target-qswkd\" (UID: \"332552b5-8c5f-4050-81f0-cf2d5377dd09\") " pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:17.468231 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.468091    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf5pt\" (UniqueName: \"kubernetes.io/projected/332552b5-8c5f-4050-81f0-cf2d5377dd09-kube-api-access-mf5pt\") pod \"network-check-target-qswkd\" (UID: \"332552b5-8c5f-4050-81f0-cf2d5377dd09\") " pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:17.619718 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.619678    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9ct5"
Sep 10 15:42:17.656834 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.656792    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:17.657020 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.656878    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:17.657174 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.657152    2527 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:42:17.657262 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.657175    2527 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67ddd8b586-7pt28: secret "image-registry-tls" not found
Sep 10 15:42:17.657262 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.657241    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls podName:abc19d51-c8e1-4f57-88c8-2004ec54c4bb nodeName:}" failed. No retries permitted until 2026-09-10 15:42:21.657218407 +0000 UTC m=+37.573343268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls") pod "image-registry-67ddd8b586-7pt28" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb") : secret "image-registry-tls" not found
Sep 10 15:42:17.657765 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.657744    2527 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:17.657921 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.657805    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls podName:137e05f8-fbb8-45bb-af7e-418ecd5f1f51 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:21.657786221 +0000 UTC m=+37.573911079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-tggpc" (UID: "137e05f8-fbb8-45bb-af7e-418ecd5f1f51") : secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:17.671955 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.671610    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:17.758689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.757708    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:17.758689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.757793    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:17.758689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.757859    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:17.758689 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.758022    2527 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:42:17.758689 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.758086    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls podName:909c47d6-65aa-4f4f-bfc3-da05c15bc3c9 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:21.75806778 +0000 UTC m=+37.674192642 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls") pod "dns-default-wnlqg" (UID: "909c47d6-65aa-4f4f-bfc3-da05c15bc3c9") : secret "dns-default-metrics-tls" not found
Sep 10 15:42:17.758689 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.758411    2527 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 15:42:17.758689 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.758464    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:21.758448239 +0000 UTC m=+37.674573098 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : secret "router-metrics-certs-default" not found
Sep 10 15:42:17.758689 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.758541    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:21.758528365 +0000 UTC m=+37.674653224 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : configmap references non-existent config key: service-ca.crt
Sep 10 15:42:17.858837 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.858789    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:17.859070 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.858893    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:17.859149 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.859118    2527 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 15:42:17.859210 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.859197    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls podName:83488306-a407-4c7f-9309-d7d47955d038 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:21.859177187 +0000 UTC m=+37.775302062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qw66j" (UID: "83488306-a407-4c7f-9309-d7d47955d038") : secret "samples-operator-tls" not found
Sep 10 15:42:17.859503 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.859408    2527 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 15:42:17.859503 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:17.859471    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert podName:078c1103-50b4-4a77-ba94-0d1e164c5b41 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:21.859454226 +0000 UTC m=+37.775579097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d47zd" (UID: "078c1103-50b4-4a77-ba94-0d1e164c5b41") : secret "networking-console-plugin-cert" not found
Sep 10 15:42:17.881084 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.880166    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj" event={"ID":"c8fd6930-9ef8-4269-8e75-370c3f2d4deb","Type":"ContainerStarted","Data":"9a802aecf408e18f79482d37a1f721b12c64d53c910cb39c8348db98dd21594a"}
Sep 10 15:42:17.981359 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:17.981320    2527 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-v96j6_ca288a4f-6d91-475a-bdd0-22fb9c6c8361/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:18.086741 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:18.086592    2527 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-v96j6_ca288a4f-6d91-475a-bdd0-22fb9c6c8361/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:18.782339 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:18.782219    2527 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-v96j6_ca288a4f-6d91-475a-bdd0-22fb9c6c8361/konnectivity-agent/0.log" limit=1024
Sep 10 15:42:21.700179 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:21.700117    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:21.700626 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:21.700223    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:21.700626 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.700295    2527 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:21.700626 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.700369    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls podName:137e05f8-fbb8-45bb-af7e-418ecd5f1f51 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:29.700346633 +0000 UTC m=+45.616471498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-tggpc" (UID: "137e05f8-fbb8-45bb-af7e-418ecd5f1f51") : secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:21.700626 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.700397    2527 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:42:21.700626 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.700421    2527 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67ddd8b586-7pt28: secret "image-registry-tls" not found
Sep 10 15:42:21.700626 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.700483    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls podName:abc19d51-c8e1-4f57-88c8-2004ec54c4bb nodeName:}" failed. No retries permitted until 2026-09-10 15:42:29.700463634 +0000 UTC m=+45.616588506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls") pod "image-registry-67ddd8b586-7pt28" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb") : secret "image-registry-tls" not found
Sep 10 15:42:21.801317 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:21.801277    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:21.801505 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:21.801340    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:21.801505 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:21.801386    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:21.801505 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.801482    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:29.801457481 +0000 UTC m=+45.717582353 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : configmap references non-existent config key: service-ca.crt
Sep 10 15:42:21.801673 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.801513    2527 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:42:21.801673 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.801514    2527 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 15:42:21.801673 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.801563    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls podName:909c47d6-65aa-4f4f-bfc3-da05c15bc3c9 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:29.801549449 +0000 UTC m=+45.717674325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls") pod "dns-default-wnlqg" (UID: "909c47d6-65aa-4f4f-bfc3-da05c15bc3c9") : secret "dns-default-metrics-tls" not found
Sep 10 15:42:21.801673 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.801609    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:29.801591606 +0000 UTC m=+45.717716479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : secret "router-metrics-certs-default" not found
Sep 10 15:42:21.902226 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:21.902192    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:21.902426 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:21.902285    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:21.902426 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.902338    2527 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 15:42:21.902426 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.902403    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert podName:078c1103-50b4-4a77-ba94-0d1e164c5b41 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:29.90238472 +0000 UTC m=+45.818509591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d47zd" (UID: "078c1103-50b4-4a77-ba94-0d1e164c5b41") : secret "networking-console-plugin-cert" not found
Sep 10 15:42:21.902543 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.902451    2527 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 15:42:21.902543 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:21.902488    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls podName:83488306-a407-4c7f-9309-d7d47955d038 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:29.902477816 +0000 UTC m=+45.818602673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qw66j" (UID: "83488306-a407-4c7f-9309-d7d47955d038") : secret "samples-operator-tls" not found
Sep 10 15:42:29.775904 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:29.775807    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:29.775904 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:29.775882    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:29.776492 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.775978    2527 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:29.776492 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.776009    2527 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 15:42:29.776492 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.776024    2527 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67ddd8b586-7pt28: secret "image-registry-tls" not found
Sep 10 15:42:29.776492 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.776058    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls podName:137e05f8-fbb8-45bb-af7e-418ecd5f1f51 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:45.776040814 +0000 UTC m=+61.692165671 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-tggpc" (UID: "137e05f8-fbb8-45bb-af7e-418ecd5f1f51") : secret "cluster-monitoring-operator-tls" not found
Sep 10 15:42:29.776492 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.776074    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls podName:abc19d51-c8e1-4f57-88c8-2004ec54c4bb nodeName:}" failed. No retries permitted until 2026-09-10 15:42:45.77606726 +0000 UTC m=+61.692192117 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls") pod "image-registry-67ddd8b586-7pt28" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb") : secret "image-registry-tls" not found
Sep 10 15:42:29.877252 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:29.877209    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:29.877451 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:29.877286    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:29.877451 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:29.877348    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:29.877451 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.877389    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:45.877366454 +0000 UTC m=+61.793491322 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : configmap references non-existent config key: service-ca.crt
Sep 10 15:42:29.877451 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.877435    2527 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 15:42:29.877451 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.877441    2527 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 15:42:29.877674 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.877499    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls podName:909c47d6-65aa-4f4f-bfc3-da05c15bc3c9 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:45.877485761 +0000 UTC m=+61.793610623 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls") pod "dns-default-wnlqg" (UID: "909c47d6-65aa-4f4f-bfc3-da05c15bc3c9") : secret "dns-default-metrics-tls" not found
Sep 10 15:42:29.877674 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.877517    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs podName:e6549d43-a327-43b8-b3a9-0c71532c4985 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:45.877507786 +0000 UTC m=+61.793632649 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs") pod "router-default-5cf6fc6776-qgwn4" (UID: "e6549d43-a327-43b8-b3a9-0c71532c4985") : secret "router-metrics-certs-default" not found
Sep 10 15:42:29.978074 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:29.978025    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:29.978277 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:29.978151    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:29.978277 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.978197    2527 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 15:42:29.978390 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.978290    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert podName:078c1103-50b4-4a77-ba94-0d1e164c5b41 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:45.978271831 +0000 UTC m=+61.894396705 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-d47zd" (UID: "078c1103-50b4-4a77-ba94-0d1e164c5b41") : secret "networking-console-plugin-cert" not found
Sep 10 15:42:29.978390 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.978381    2527 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 15:42:29.978487 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:29.978451    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls podName:83488306-a407-4c7f-9309-d7d47955d038 nodeName:}" failed. No retries permitted until 2026-09-10 15:42:45.9784322 +0000 UTC m=+61.894557072 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qw66j" (UID: "83488306-a407-4c7f-9309-d7d47955d038") : secret "samples-operator-tls" not found
Sep 10 15:42:30.878765 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.878227    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-58bxj" podStartSLOduration=38.401716144 podStartE2EDuration="40.878201102s" totalImagesPullingTime="2.476484958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:17.896657577 +0000 UTC m=+33.812782483" watchObservedRunningTime="2026-09-10 15:42:30.878201102 +0000 UTC m=+46.794325982"
Sep 10 15:42:30.878765 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.878652    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qswkd"]
Sep 10 15:42:30.887345 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:30.886964    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332552b5_8c5f_4050_81f0_cf2d5377dd09.slice/crio-5ff0e701896ca8464bbbab45afb9ac85e722cde1acb9d0f9d8ea4565e50dfef9 WatchSource:0}: Error finding container 5ff0e701896ca8464bbbab45afb9ac85e722cde1acb9d0f9d8ea4565e50dfef9: Status 404 returned error can't find the container with id 5ff0e701896ca8464bbbab45afb9ac85e722cde1acb9d0f9d8ea4565e50dfef9
Sep 10 15:42:30.911816 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.911583    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9ct5"]
Sep 10 15:42:30.919411 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:30.919375    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d7f839_f1d5_4657_aa6c_5b19b248622b.slice/crio-40a0de91fb9572843276e1d0f61698782eee7cadbf49ad606159ec730d438e64 WatchSource:0}: Error finding container 40a0de91fb9572843276e1d0f61698782eee7cadbf49ad606159ec730d438e64: Status 404 returned error can't find the container with id 40a0de91fb9572843276e1d0f61698782eee7cadbf49ad606159ec730d438e64
Sep 10 15:42:30.929969 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.929928    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc" event={"ID":"ee34d318-9263-4aeb-a95f-316447f94aa2","Type":"ContainerStarted","Data":"b149a36577c66c5ca7fe7c18184a693972cf716fd717dd59ded92ca382b41e55"}
Sep 10 15:42:30.937825 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.937320    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t" event={"ID":"8101fd3c-1dcc-4099-88af-6581442ee257","Type":"ContainerStarted","Data":"30ce2c209f4b1e256702c08ec5354fd92a32f5d7c194a0b931b3f7960345a948"}
Sep 10 15:42:30.941161 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.939955    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z9ct5" event={"ID":"f0d7f839-f1d5-4657-aa6c-5b19b248622b","Type":"ContainerStarted","Data":"40a0de91fb9572843276e1d0f61698782eee7cadbf49ad606159ec730d438e64"}
Sep 10 15:42:30.942528 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.942498    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4" event={"ID":"2b7b241e-708b-40bb-ba3d-52c0fee5e425","Type":"ContainerStarted","Data":"aa75eb5aa0be8f34c91c71aae5140b4cfaa971d018b66263cadc0b70970257d5"}
Sep 10 15:42:30.946785 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.946764    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/0.log"
Sep 10 15:42:30.946959 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.946941    2527 generic.go:356] "Generic (PLEG): container finished" podID="d30a715d-e7b9-4e9e-b311-2675c91df3df" containerID="8acaaf0c77bf70fd705b70baca7bde31c4e45eca959c70bf5bab268b20197e39" exitCode=255
Sep 10 15:42:30.947641 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.947053    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-595p9" event={"ID":"d30a715d-e7b9-4e9e-b311-2675c91df3df","Type":"ContainerDied","Data":"8acaaf0c77bf70fd705b70baca7bde31c4e45eca959c70bf5bab268b20197e39"}
Sep 10 15:42:30.952121 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.947483    2527 scope.go:117] "RemoveContainer" containerID="8acaaf0c77bf70fd705b70baca7bde31c4e45eca959c70bf5bab268b20197e39"
Sep 10 15:42:30.952121 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.949613    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc" podStartSLOduration=25.045593686 podStartE2EDuration="40.949592077s" totalImagesPullingTime="15.903998391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:30.946452518 +0000 UTC m=+46.862577398" watchObservedRunningTime="2026-09-10 15:42:30.949592077 +0000 UTC m=+46.865716957"
Sep 10 15:42:30.958016 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:30.957609    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qswkd" event={"ID":"332552b5-8c5f-4050-81f0-cf2d5377dd09","Type":"ContainerStarted","Data":"5ff0e701896ca8464bbbab45afb9ac85e722cde1acb9d0f9d8ea4565e50dfef9"}
Sep 10 15:42:31.003088 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.001471    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t" podStartSLOduration=25.031824125 podStartE2EDuration="41.001450826s" totalImagesPullingTime="15.969626701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:30.968463838 +0000 UTC m=+46.884588718" watchObservedRunningTime="2026-09-10 15:42:31.001450826 +0000 UTC m=+46.917575707"
Sep 10 15:42:31.003088 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.002418    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4" podStartSLOduration=25.126234081 podStartE2EDuration="41.002403123s" totalImagesPullingTime="15.876169042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:31.000352643 +0000 UTC m=+46.916477525" watchObservedRunningTime="2026-09-10 15:42:31.002403123 +0000 UTC m=+46.918528005"
Sep 10 15:42:31.927364 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.925660    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gqclh"]
Sep 10 15:42:31.942752 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.941545    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gqclh"]
Sep 10 15:42:31.942752 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.941687    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:31.945577 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.944077    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5dvnv\""
Sep 10 15:42:31.945577 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.944605    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 15:42:31.945577 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.944669    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 15:42:31.972038 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.971534    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-pxwpp" event={"ID":"ae5f449d-8e49-47ca-976a-b63650f74b6a","Type":"ContainerStarted","Data":"3d695b7102f82fd5fa9ccfceb2914b940c3b544f7d5552820580573a4983d133"}
Sep 10 15:42:31.974305 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.974272    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h5mql" event={"ID":"2120541f-3d27-4e5e-8439-374a8d237e26","Type":"ContainerStarted","Data":"204aa6def0a8b30f98ceeb4d1e30d32f03c30f5ef5ee720bd3d00f6ad0065f88"}
Sep 10 15:42:31.977663 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.977631    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6" event={"ID":"17f2efec-4d06-45ed-8f25-4c95181e821e","Type":"ContainerStarted","Data":"16108b2f3f9e4c8dd8207ff394be0516abc77bc39154750d3d98b6cd6d761dee"}
Sep 10 15:42:31.978086 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.978065    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:31.980214 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.980190    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6"
Sep 10 15:42:31.983283 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.983269    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 15:42:31.984024 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.984006    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/0.log"
Sep 10 15:42:31.984201 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.984181    2527 generic.go:356] "Generic (PLEG): container finished" podID="d30a715d-e7b9-4e9e-b311-2675c91df3df" containerID="54c5c366756b822addb2a01cc0b18816cd5a0ca185aae67e17c3b3dd8378b93c" exitCode=255
Sep 10 15:42:31.984567 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.984313    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-595p9" event={"ID":"d30a715d-e7b9-4e9e-b311-2675c91df3df","Type":"ContainerDied","Data":"54c5c366756b822addb2a01cc0b18816cd5a0ca185aae67e17c3b3dd8378b93c"}
Sep 10 15:42:31.984567 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.984471    2527 scope.go:117] "RemoveContainer" containerID="8acaaf0c77bf70fd705b70baca7bde31c4e45eca959c70bf5bab268b20197e39"
Sep 10 15:42:31.984711 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.984581    2527 scope.go:117] "RemoveContainer" containerID="54c5c366756b822addb2a01cc0b18816cd5a0ca185aae67e17c3b3dd8378b93c"
Sep 10 15:42:31.984815 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:31.984789    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-595p9_openshift-console-operator(d30a715d-e7b9-4e9e-b311-2675c91df3df)\"" pod="openshift-console-operator/console-operator-85b454874b-595p9" podUID="d30a715d-e7b9-4e9e-b311-2675c91df3df"
Sep 10 15:42:31.988932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.988485    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-pxwpp" podStartSLOduration=17.172495203 podStartE2EDuration="32.988467955s" totalImagesPullingTime="15.815972752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:31.988372925 +0000 UTC m=+47.904497827" watchObservedRunningTime="2026-09-10 15:42:31.988467955 +0000 UTC m=+47.904592837"
Sep 10 15:42:31.991853 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.991835    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" event={"ID":"67f934b5-d4b9-49af-a439-0e77d6890343","Type":"ContainerStarted","Data":"7332099e25ad94ee8130edd0aebf85531c90b3b47f816af550fb2ae9bb51de17"}
Sep 10 15:42:31.991970 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.991959    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" event={"ID":"67f934b5-d4b9-49af-a439-0e77d6890343","Type":"ContainerStarted","Data":"35380476af268c017e9aa6e7501b2899d00bfdc5da0daa272d32b1d10ddc404b"}
Sep 10 15:42:31.992043 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.992034    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" event={"ID":"67f934b5-d4b9-49af-a439-0e77d6890343","Type":"ContainerStarted","Data":"296bfbbec36bfa1daedce688fd443fd3ba3454382e611e7d8cb73af614e64ca5"}
Sep 10 15:42:31.994692 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.994636    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qswkd" event={"ID":"332552b5-8c5f-4050-81f0-cf2d5377dd09","Type":"ContainerStarted","Data":"8dd950c0494237f38ace7e3c9016c782627601003861a16501a0f337c6ba67b8"}
Sep 10 15:42:31.994796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.994744    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:42:31.999022 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.998993    2527 generic.go:356] "Generic (PLEG): container finished" podID="a40c2ed6-ebbb-4c90-919c-23172dca982e" containerID="bb44334db978dc4340851826b811b83c0813fccd4e649db5561bb198c7c0c986" exitCode=0
Sep 10 15:42:31.999290 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:31.999269    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps5pl" event={"ID":"a40c2ed6-ebbb-4c90-919c-23172dca982e","Type":"ContainerDied","Data":"bb44334db978dc4340851826b811b83c0813fccd4e649db5561bb198c7c0c986"}
Sep 10 15:42:32.002892 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.002835    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.003009 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.002986    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdbxd\" (UniqueName: \"kubernetes.io/projected/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-api-access-jdbxd\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.003112 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.003093    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-data-volume\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.003216 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.003199    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-crio-socket\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.003793 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.003599    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.026109 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.025599    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7fb8f8b574-d7mw6" podStartSLOduration=28.133899987 podStartE2EDuration="44.025573818s" totalImagesPullingTime="15.891673831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:32.023708744 +0000 UTC m=+47.939833625" watchObservedRunningTime="2026-09-10 15:42:32.025573818 +0000 UTC m=+47.941698702"
Sep 10 15:42:32.044096 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.044048    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-h5mql" podStartSLOduration=3.215856307 podStartE2EDuration="19.044029685s" totalImagesPullingTime="15.828173378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:32.043184074 +0000 UTC m=+47.959308978" watchObservedRunningTime="2026-09-10 15:42:32.044029685 +0000 UTC m=+47.960154564"
Sep 10 15:42:32.081070 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.080808    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qswkd" podStartSLOduration=47.080792078 podStartE2EDuration="47.080792078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:32.080746922 +0000 UTC m=+47.996871802" watchObservedRunningTime="2026-09-10 15:42:32.080792078 +0000 UTC m=+47.996916958"
Sep 10 15:42:32.105048 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.104884    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" podStartSLOduration=27.105433756 podStartE2EDuration="43.104858166s" totalImagesPullingTime="15.99942441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:32.103213525 +0000 UTC m=+48.019338419" watchObservedRunningTime="2026-09-10 15:42:32.104858166 +0000 UTC m=+48.020983048"
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.106163    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.106207    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdbxd\" (UniqueName: \"kubernetes.io/projected/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-api-access-jdbxd\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.106414    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-data-volume\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.106508    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-crio-socket\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.106648    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:32.107000    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:32.107062    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:32.607044743 +0000 UTC m=+48.523169622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:32.107361    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:32.107416    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:32.607399116 +0000 UTC m=+48.523523979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : configmap "kube-rbac-proxy" not found
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.107541    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-crio-socket\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.108638 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.108253    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-data-volume\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.120942 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.120877    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdbxd\" (UniqueName: \"kubernetes.io/projected/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-api-access-jdbxd\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.307563 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.307520    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt"]
Sep 10 15:42:32.310090 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.310063    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt"
Sep 10 15:42:32.312481 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.312451    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 15:42:32.312716 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.312459    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bzjlr\""
Sep 10 15:42:32.312910 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.312892    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 15:42:32.320598 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.320531    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt"]
Sep 10 15:42:32.410419 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.410379    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg94k\" (UniqueName: \"kubernetes.io/projected/3df79d94-eaf2-4a4a-a368-8906ea79d488-kube-api-access-tg94k\") pod \"migrator-fdbb8fdff-8psxt\" (UID: \"3df79d94-eaf2-4a4a-a368-8906ea79d488\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt"
Sep 10 15:42:32.511915 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.511825    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg94k\" (UniqueName: \"kubernetes.io/projected/3df79d94-eaf2-4a4a-a368-8906ea79d488-kube-api-access-tg94k\") pod \"migrator-fdbb8fdff-8psxt\" (UID: \"3df79d94-eaf2-4a4a-a368-8906ea79d488\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt"
Sep 10 15:42:32.523254 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.523220    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg94k\" (UniqueName: \"kubernetes.io/projected/3df79d94-eaf2-4a4a-a368-8906ea79d488-kube-api-access-tg94k\") pod \"migrator-fdbb8fdff-8psxt\" (UID: \"3df79d94-eaf2-4a4a-a368-8906ea79d488\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt"
Sep 10 15:42:32.613093 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.613054    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.613330 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.613183    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:32.613330 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:32.613256    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:32.613330 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:32.613313    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:42:32.613466 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:32.613358    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:33.613334999 +0000 UTC m=+49.529459859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:32.613466 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:32.613381    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:33.613369229 +0000 UTC m=+49.529494086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : configmap "kube-rbac-proxy" not found
Sep 10 15:42:32.622393 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.622367    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt"
Sep 10 15:42:32.909566 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.909537    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4djmx_54d5a997-3142-4442-a345-298d70f46804/dns-node-resolver/0.log"
Sep 10 15:42:32.937696 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:32.937664    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt"]
Sep 10 15:42:32.942391 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:32.942355    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df79d94_eaf2_4a4a_a368_8906ea79d488.slice/crio-e4d3545eee2d054cdb8b8d9a5195e1cb5c5650f28d136c7ef69a85b709adafa8 WatchSource:0}: Error finding container e4d3545eee2d054cdb8b8d9a5195e1cb5c5650f28d136c7ef69a85b709adafa8: Status 404 returned error can't find the container with id e4d3545eee2d054cdb8b8d9a5195e1cb5c5650f28d136c7ef69a85b709adafa8
Sep 10 15:42:33.007491 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:33.007453    2527 generic.go:356] "Generic (PLEG): container finished" podID="a40c2ed6-ebbb-4c90-919c-23172dca982e" containerID="009a549617681d730beeee36fc64ef99e4aa67173b152492139363e08af475e9" exitCode=0
Sep 10 15:42:33.007742 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:33.007540    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps5pl" event={"ID":"a40c2ed6-ebbb-4c90-919c-23172dca982e","Type":"ContainerDied","Data":"009a549617681d730beeee36fc64ef99e4aa67173b152492139363e08af475e9"}
Sep 10 15:42:33.015050 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:33.014996    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx" event={"ID":"3e839405-30b3-4105-80d1-3d29a1fe8683","Type":"ContainerStarted","Data":"0c4bb65fa4695b110fd9ba6126881d320c261d2285256d34f3620ffb4fe64672"}
Sep 10 15:42:33.016554 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:33.016519    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt" event={"ID":"3df79d94-eaf2-4a4a-a368-8906ea79d488","Type":"ContainerStarted","Data":"e4d3545eee2d054cdb8b8d9a5195e1cb5c5650f28d136c7ef69a85b709adafa8"}
Sep 10 15:42:33.018443 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:33.018415    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 15:42:33.019492 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:33.019468    2527 scope.go:117] "RemoveContainer" containerID="54c5c366756b822addb2a01cc0b18816cd5a0ca185aae67e17c3b3dd8378b93c"
Sep 10 15:42:33.019892 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:33.019869    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-595p9_openshift-console-operator(d30a715d-e7b9-4e9e-b311-2675c91df3df)\"" pod="openshift-console-operator/console-operator-85b454874b-595p9" podUID="d30a715d-e7b9-4e9e-b311-2675c91df3df"
Sep 10 15:42:33.052095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:33.051418    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-586bb8f644-wx5wx" podStartSLOduration=27.224321893 podStartE2EDuration="45.051394754s" totalImagesPullingTime="17.827072861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-10 15:42:33.050018609 +0000 UTC m=+48.966143490" watchObservedRunningTime="2026-09-10 15:42:33.051394754 +0000 UTC m=+48.967519637"
Sep 10 15:42:33.627204 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:33.626468    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:33.627204 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:33.626598    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:33.627204 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:33.626768    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:33.627204 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:33.626843    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:35.6268236 +0000 UTC m=+51.542948459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:33.627204 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:33.626861    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:42:33.627204 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:33.626901    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:35.626890351 +0000 UTC m=+51.543015223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : configmap "kube-rbac-proxy" not found
Sep 10 15:42:34.029062 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.029024    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ps5pl" event={"ID":"a40c2ed6-ebbb-4c90-919c-23172dca982e","Type":"ContainerStarted","Data":"84fe786395227944d00756dd5d6567d4a040341daced01100efca9f85ac75068"}
Sep 10 15:42:34.055222 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.055166    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ps5pl" podStartSLOduration=7.73951133 podStartE2EDuration="49.055146766s" totalImagesPullingTime="41.315635436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:34.054045476 +0000 UTC m=+49.970170376" watchObservedRunningTime="2026-09-10 15:42:34.055146766 +0000 UTC m=+49.971271636"
Sep 10 15:42:34.104881 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.104848    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xfqhc_193f19ff-ce68-42c7-929f-f26b2c5edde7/node-ca/0.log"
Sep 10 15:42:34.283696 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.283579    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:34.283696 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.283650    2527 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:34.284093 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.284064    2527 scope.go:117] "RemoveContainer" containerID="54c5c366756b822addb2a01cc0b18816cd5a0ca185aae67e17c3b3dd8378b93c"
Sep 10 15:42:34.284331 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:34.284291    2527 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-595p9_openshift-console-operator(d30a715d-e7b9-4e9e-b311-2675c91df3df)\"" pod="openshift-console-operator/console-operator-85b454874b-595p9" podUID="d30a715d-e7b9-4e9e-b311-2675c91df3df"
Sep 10 15:42:34.475499 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.475327    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-dqr9v"]
Sep 10 15:42:34.478812 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.478785    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.481116 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.480898    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-v8gk5\""
Sep 10 15:42:34.481116 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.480898    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 15:42:34.481116 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.481059    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 15:42:34.481362 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.481225    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 15:42:34.481362 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.481253    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 15:42:34.488868 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.487901    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-dqr9v"]
Sep 10 15:42:34.539149 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.539037    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfk58\" (UniqueName: \"kubernetes.io/projected/1e8ca618-250a-4998-85d3-8fa76362a685-kube-api-access-jfk58\") pod \"service-ca-75d99c484-dqr9v\" (UID: \"1e8ca618-250a-4998-85d3-8fa76362a685\") " pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.539149 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.539088    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1e8ca618-250a-4998-85d3-8fa76362a685-signing-cabundle\") pod \"service-ca-75d99c484-dqr9v\" (UID: \"1e8ca618-250a-4998-85d3-8fa76362a685\") " pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.539339 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.539244    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1e8ca618-250a-4998-85d3-8fa76362a685-signing-key\") pod \"service-ca-75d99c484-dqr9v\" (UID: \"1e8ca618-250a-4998-85d3-8fa76362a685\") " pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.639726 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.639683    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1e8ca618-250a-4998-85d3-8fa76362a685-signing-key\") pod \"service-ca-75d99c484-dqr9v\" (UID: \"1e8ca618-250a-4998-85d3-8fa76362a685\") " pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.639935 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.639819    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfk58\" (UniqueName: \"kubernetes.io/projected/1e8ca618-250a-4998-85d3-8fa76362a685-kube-api-access-jfk58\") pod \"service-ca-75d99c484-dqr9v\" (UID: \"1e8ca618-250a-4998-85d3-8fa76362a685\") " pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.639935 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.639867    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1e8ca618-250a-4998-85d3-8fa76362a685-signing-cabundle\") pod \"service-ca-75d99c484-dqr9v\" (UID: \"1e8ca618-250a-4998-85d3-8fa76362a685\") " pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.640624 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.640599    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1e8ca618-250a-4998-85d3-8fa76362a685-signing-cabundle\") pod \"service-ca-75d99c484-dqr9v\" (UID: \"1e8ca618-250a-4998-85d3-8fa76362a685\") " pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.642554 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.642517    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1e8ca618-250a-4998-85d3-8fa76362a685-signing-key\") pod \"service-ca-75d99c484-dqr9v\" (UID: \"1e8ca618-250a-4998-85d3-8fa76362a685\") " pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.648666 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.648638    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfk58\" (UniqueName: \"kubernetes.io/projected/1e8ca618-250a-4998-85d3-8fa76362a685-kube-api-access-jfk58\") pod \"service-ca-75d99c484-dqr9v\" (UID: \"1e8ca618-250a-4998-85d3-8fa76362a685\") " pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:34.701966 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.701928    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h5mql_2120541f-3d27-4e5e-8439-374a8d237e26/serve-healthcheck-canary/0.log"
Sep 10 15:42:34.792862 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:34.792772    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-dqr9v"
Sep 10 15:42:35.034780 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:35.034743    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt" event={"ID":"3df79d94-eaf2-4a4a-a368-8906ea79d488","Type":"ContainerStarted","Data":"a1b623b6f31f7b3e0174de08a3a0bd2d254eb702a8493138588aa9c70bb7caa9"}
Sep 10 15:42:35.035252 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:35.034839    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt" event={"ID":"3df79d94-eaf2-4a4a-a368-8906ea79d488","Type":"ContainerStarted","Data":"e28c4d6562f4846387a1fdbc57fa13bb23838fb20e676b74fe6839723c6d520c"}
Sep 10 15:42:35.052314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:35.052209    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8psxt" podStartSLOduration=1.678714191 podStartE2EDuration="3.052191112s" totalImagesPullingTime="1.373476921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:35.051280873 +0000 UTC m=+50.967405753" watchObservedRunningTime="2026-09-10 15:42:35.052191112 +0000 UTC m=+50.968315994"
Sep 10 15:42:35.076637 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:35.076595    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-dqr9v"]
Sep 10 15:42:35.081409 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:35.081374    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8ca618_250a_4998_85d3_8fa76362a685.slice/crio-de5d95deec042941c6f8013a2dea7bb9a1e6b5213c20cca18633e694e3e37718 WatchSource:0}: Error finding container de5d95deec042941c6f8013a2dea7bb9a1e6b5213c20cca18633e694e3e37718: Status 404 returned error can't find the container with id de5d95deec042941c6f8013a2dea7bb9a1e6b5213c20cca18633e694e3e37718
Sep 10 15:42:35.651381 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:35.651338    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:35.651578 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:35.651494    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:42:35.651643 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:35.651576    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:39.651554274 +0000 UTC m=+55.567679138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : configmap "kube-rbac-proxy" not found
Sep 10 15:42:35.651643 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:35.651457    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:35.651643 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:35.651599    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:35.651813 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:35.651659    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:39.651641589 +0000 UTC m=+55.567766447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:36.041336 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:36.041294    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-dqr9v" event={"ID":"1e8ca618-250a-4998-85d3-8fa76362a685","Type":"ContainerStarted","Data":"6302e0768cba727cae288198604f22cb074f656d9be1b31b3f0d90b54fa8fd86"}
Sep 10 15:42:36.041791 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:36.041351    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-dqr9v" event={"ID":"1e8ca618-250a-4998-85d3-8fa76362a685","Type":"ContainerStarted","Data":"de5d95deec042941c6f8013a2dea7bb9a1e6b5213c20cca18633e694e3e37718"}
Sep 10 15:42:36.062581 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:36.062497    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-dqr9v" podStartSLOduration=2.062476659 podStartE2EDuration="2.062476659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:36.062246576 +0000 UTC m=+51.978371456" watchObservedRunningTime="2026-09-10 15:42:36.062476659 +0000 UTC m=+51.978601572"
Sep 10 15:42:38.054593 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:38.054558    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z9ct5" event={"ID":"f0d7f839-f1d5-4657-aa6c-5b19b248622b","Type":"ContainerStarted","Data":"6d23313cce1667d327f4b3ecc459119db2d81d2773b4ffd63d3a6a05509e01e7"}
Sep 10 15:42:38.090468 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:38.090400    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z9ct5" podStartSLOduration=46.347915757 podStartE2EDuration="53.090381294s" totalImagesPullingTime="6.742465537s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:38.087715143 +0000 UTC m=+54.003840022" watchObservedRunningTime="2026-09-10 15:42:38.090381294 +0000 UTC m=+54.006506181"
Sep 10 15:42:39.693110 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:39.693058    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:39.693605 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:39.693216    2527 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 15:42:39.693605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:39.693281    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:39.693605 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:39.693290    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:47.693273313 +0000 UTC m=+63.609398175 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : configmap "kube-rbac-proxy" not found
Sep 10 15:42:39.693605 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:39.693417    2527 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:39.693605 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:42:39.693472    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls podName:ee6b2a6f-4882-4cce-96cf-a8225b5b864e nodeName:}" failed. No retries permitted until 2026-09-10 15:42:47.693457149 +0000 UTC m=+63.609582007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gqclh" (UID: "ee6b2a6f-4882-4cce-96cf-a8225b5b864e") : secret "insights-runtime-extractor-tls" not found
Sep 10 15:42:40.811771 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:40.811741    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb2zb"
Sep 10 15:42:45.851542 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.851503    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:45.852169 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.851565    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:45.854068 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.854040    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/137e05f8-fbb8-45bb-af7e-418ecd5f1f51-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-tggpc\" (UID: \"137e05f8-fbb8-45bb-af7e-418ecd5f1f51\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:45.854203 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.854050    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"image-registry-67ddd8b586-7pt28\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") " pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:45.952374 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.952337    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:45.952544 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.952393    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:45.952544 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.952429    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:45.952997 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.952962    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6549d43-a327-43b8-b3a9-0c71532c4985-service-ca-bundle\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:45.954811 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.954778    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/909c47d6-65aa-4f4f-bfc3-da05c15bc3c9-metrics-tls\") pod \"dns-default-wnlqg\" (UID: \"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9\") " pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:45.954905 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:45.954856    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6549d43-a327-43b8-b3a9-0c71532c4985-metrics-certs\") pod \"router-default-5cf6fc6776-qgwn4\" (UID: \"e6549d43-a327-43b8-b3a9-0c71532c4985\") " pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:46.047191 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.047155    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hck2z\""
Sep 10 15:42:46.053444 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.053418    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:46.053545 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.053470    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:46.054976 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.054948    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:46.056082 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.056060    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/078c1103-50b4-4a77-ba94-0d1e164c5b41-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-d47zd\" (UID: \"078c1103-50b4-4a77-ba94-0d1e164c5b41\") " pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:46.056201 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.056116    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/83488306-a407-4c7f-9309-d7d47955d038-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qw66j\" (UID: \"83488306-a407-4c7f-9309-d7d47955d038\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:46.064009 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.063973    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tc2md\""
Sep 10 15:42:46.072753 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.072721    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"
Sep 10 15:42:46.107603 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.107581    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d968c\""
Sep 10 15:42:46.116392 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.116368    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:46.146309 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.146270    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-brxch\""
Sep 10 15:42:46.157532 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.157253    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:46.212875 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.212641    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6qzgp\""
Sep 10 15:42:46.220612 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.220564    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"
Sep 10 15:42:46.261003 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.260970    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-njtv5\""
Sep 10 15:42:46.269475 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.269438    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd"
Sep 10 15:42:46.382532 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.382415    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67ddd8b586-7pt28"]
Sep 10 15:42:46.385059 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:46.385016    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc19d51_c8e1_4f57_88c8_2004ec54c4bb.slice/crio-edd1d8bbf7bf57cc1db73b851d9244b04d90597310a2649777708712ee7e79db WatchSource:0}: Error finding container edd1d8bbf7bf57cc1db73b851d9244b04d90597310a2649777708712ee7e79db: Status 404 returned error can't find the container with id edd1d8bbf7bf57cc1db73b851d9244b04d90597310a2649777708712ee7e79db
Sep 10 15:42:46.407009 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.406974    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc"]
Sep 10 15:42:46.410038 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:46.410007    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod137e05f8_fbb8_45bb_af7e_418ecd5f1f51.slice/crio-2c6caebd23283d2cd725ff0d2fee52dbbb7de974f85ad10e32aa7e43198e61fe WatchSource:0}: Error finding container 2c6caebd23283d2cd725ff0d2fee52dbbb7de974f85ad10e32aa7e43198e61fe: Status 404 returned error can't find the container with id 2c6caebd23283d2cd725ff0d2fee52dbbb7de974f85ad10e32aa7e43198e61fe
Sep 10 15:42:46.434711 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.434682    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5cf6fc6776-qgwn4"]
Sep 10 15:42:46.440244 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:46.440217    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6549d43_a327_43b8_b3a9_0c71532c4985.slice/crio-daadeaed2c1dd24fb9e34d399d5fc6ab850f30c155ba8323b056fda9708e4f77 WatchSource:0}: Error finding container daadeaed2c1dd24fb9e34d399d5fc6ab850f30c155ba8323b056fda9708e4f77: Status 404 returned error can't find the container with id daadeaed2c1dd24fb9e34d399d5fc6ab850f30c155ba8323b056fda9708e4f77
Sep 10 15:42:46.466547 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.466517    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wnlqg"]
Sep 10 15:42:46.468727 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:46.468693    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909c47d6_65aa_4f4f_bfc3_da05c15bc3c9.slice/crio-459d50298f4d96aa78238070dd291b632b2b862ffada61c47455b66f57bc8501 WatchSource:0}: Error finding container 459d50298f4d96aa78238070dd291b632b2b862ffada61c47455b66f57bc8501: Status 404 returned error can't find the container with id 459d50298f4d96aa78238070dd291b632b2b862ffada61c47455b66f57bc8501
Sep 10 15:42:46.513313 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.513278    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j"]
Sep 10 15:42:46.563408 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:46.563327    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-d47zd"]
Sep 10 15:42:46.568726 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:46.568702    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078c1103_50b4_4a77_ba94_0d1e164c5b41.slice/crio-45e3829b9c73d9ab6dcb8b36e02c590669169d601d946f9f475b76b473529c0c WatchSource:0}: Error finding container 45e3829b9c73d9ab6dcb8b36e02c590669169d601d946f9f475b76b473529c0c: Status 404 returned error can't find the container with id 45e3829b9c73d9ab6dcb8b36e02c590669169d601d946f9f475b76b473529c0c
Sep 10 15:42:47.095033 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.094984    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd" event={"ID":"078c1103-50b4-4a77-ba94-0d1e164c5b41","Type":"ContainerStarted","Data":"45e3829b9c73d9ab6dcb8b36e02c590669169d601d946f9f475b76b473529c0c"}
Sep 10 15:42:47.096592 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.096559    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5cf6fc6776-qgwn4" event={"ID":"e6549d43-a327-43b8-b3a9-0c71532c4985","Type":"ContainerStarted","Data":"049847250ffde8adc3f2ea4bca1a923e12e436ac55a5d7224e375901675df57e"}
Sep 10 15:42:47.096751 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.096612    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5cf6fc6776-qgwn4" event={"ID":"e6549d43-a327-43b8-b3a9-0c71532c4985","Type":"ContainerStarted","Data":"daadeaed2c1dd24fb9e34d399d5fc6ab850f30c155ba8323b056fda9708e4f77"}
Sep 10 15:42:47.097867 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.097840    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wnlqg" event={"ID":"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9","Type":"ContainerStarted","Data":"459d50298f4d96aa78238070dd291b632b2b862ffada61c47455b66f57bc8501"}
Sep 10 15:42:47.099634 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.099612    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28" event={"ID":"abc19d51-c8e1-4f57-88c8-2004ec54c4bb","Type":"ContainerStarted","Data":"0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54"}
Sep 10 15:42:47.099763 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.099647    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28" event={"ID":"abc19d51-c8e1-4f57-88c8-2004ec54c4bb","Type":"ContainerStarted","Data":"edd1d8bbf7bf57cc1db73b851d9244b04d90597310a2649777708712ee7e79db"}
Sep 10 15:42:47.099763 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.099718    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:42:47.100979 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.100948    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j" event={"ID":"83488306-a407-4c7f-9309-d7d47955d038","Type":"ContainerStarted","Data":"ff2e6b2df6aa65949cea45d48772408a868ea941fe5e62f04965b0f5f999ba28"}
Sep 10 15:42:47.102179 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.102154    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc" event={"ID":"137e05f8-fbb8-45bb-af7e-418ecd5f1f51","Type":"ContainerStarted","Data":"2c6caebd23283d2cd725ff0d2fee52dbbb7de974f85ad10e32aa7e43198e61fe"}
Sep 10 15:42:47.115741 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.115689    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5cf6fc6776-qgwn4" podStartSLOduration=57.115673133 podStartE2EDuration="57.115673133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:47.114828504 +0000 UTC m=+63.030953385" watchObservedRunningTime="2026-09-10 15:42:47.115673133 +0000 UTC m=+63.031798003"
Sep 10 15:42:47.116648 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.116625    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:47.119511 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.119489    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:47.134632 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.134587    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28" podStartSLOduration=62.134571789 podStartE2EDuration="1m2.134571789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:47.133644836 +0000 UTC m=+63.049769742" watchObservedRunningTime="2026-09-10 15:42:47.134571789 +0000 UTC m=+63.050696668"
Sep 10 15:42:47.558749 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.558281    2527 scope.go:117] "RemoveContainer" containerID="54c5c366756b822addb2a01cc0b18816cd5a0ca185aae67e17c3b3dd8378b93c"
Sep 10 15:42:47.773855 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.773808    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:47.774055 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.773919    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:47.775038 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.774612    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:47.783885 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.780284    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee6b2a6f-4882-4cce-96cf-a8225b5b864e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gqclh\" (UID: \"ee6b2a6f-4882-4cce-96cf-a8225b5b864e\") " pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:47.863365 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.863123    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5dvnv\""
Sep 10 15:42:47.871771 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:47.871613    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gqclh"
Sep 10 15:42:48.110409 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:48.110348    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:48.113046 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:48.112475    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5cf6fc6776-qgwn4"
Sep 10 15:42:48.363073 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:48.362959    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gqclh"]
Sep 10 15:42:49.116112 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:49.116082    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 15:42:49.116604 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:49.116209    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-595p9" event={"ID":"d30a715d-e7b9-4e9e-b311-2675c91df3df","Type":"ContainerStarted","Data":"cd65cce2371fb94b51c62cae851da0fa2baaeb31a650a10efb87e14ccf7a3fe1"}
Sep 10 15:42:49.116737 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:49.116692    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:49.122586 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:49.122550    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-595p9"
Sep 10 15:42:49.136893 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:49.136820    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-595p9" podStartSLOduration=43.246305639 podStartE2EDuration="59.136800294s" totalImagesPullingTime="15.890494655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:49.136684775 +0000 UTC m=+65.052809655" watchObservedRunningTime="2026-09-10 15:42:49.136800294 +0000 UTC m=+65.052925173"
Sep 10 15:42:49.391127 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:49.391037    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:49.395164 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:49.393955    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/511e3fe0-d13f-413c-9877-1e056560de23-metrics-certs\") pod \"network-metrics-daemon-5c6r7\" (UID: \"511e3fe0-d13f-413c-9877-1e056560de23\") " pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:49.411621 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:49.411587    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2gdj6\""
Sep 10 15:42:49.419590 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:49.419555    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5c6r7"
Sep 10 15:42:49.918333 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:49.918288    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6b2a6f_4882_4cce_96cf_a8225b5b864e.slice/crio-320ca0428d86d8b353ce0966867ae0bae2266d349e6828e2531b75d5b1e8b0c0 WatchSource:0}: Error finding container 320ca0428d86d8b353ce0966867ae0bae2266d349e6828e2531b75d5b1e8b0c0: Status 404 returned error can't find the container with id 320ca0428d86d8b353ce0966867ae0bae2266d349e6828e2531b75d5b1e8b0c0
Sep 10 15:42:50.121715 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:50.121673    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gqclh" event={"ID":"ee6b2a6f-4882-4cce-96cf-a8225b5b864e","Type":"ContainerStarted","Data":"320ca0428d86d8b353ce0966867ae0bae2266d349e6828e2531b75d5b1e8b0c0"}
Sep 10 15:42:50.783100 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:50.783072    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5c6r7"]
Sep 10 15:42:50.788699 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:50.788663    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511e3fe0_d13f_413c_9877_1e056560de23.slice/crio-948957f8fa5c7225eb79c1a355a7a419f761460907904721e93b66622f202088 WatchSource:0}: Error finding container 948957f8fa5c7225eb79c1a355a7a419f761460907904721e93b66622f202088: Status 404 returned error can't find the container with id 948957f8fa5c7225eb79c1a355a7a419f761460907904721e93b66622f202088
Sep 10 15:42:51.128759 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.128710    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc" event={"ID":"137e05f8-fbb8-45bb-af7e-418ecd5f1f51","Type":"ContainerStarted","Data":"360fe3551495cc3f11c134a5d508476d046a19fe0c5bf05dcd69e7f5f9716bf7"}
Sep 10 15:42:51.131338 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.131305    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gqclh" event={"ID":"ee6b2a6f-4882-4cce-96cf-a8225b5b864e","Type":"ContainerStarted","Data":"c1e5af514833bb5d0b01d43113aa500a2ac206584c191bcb7dde1b9736b47587"}
Sep 10 15:42:51.133705 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.133670    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd" event={"ID":"078c1103-50b4-4a77-ba94-0d1e164c5b41","Type":"ContainerStarted","Data":"735d5d72339601d10bd22a1cb0c6c6edc748df50fa54e82a84bf00f3ec79c0b8"}
Sep 10 15:42:51.135259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.135225    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5c6r7" event={"ID":"511e3fe0-d13f-413c-9877-1e056560de23","Type":"ContainerStarted","Data":"948957f8fa5c7225eb79c1a355a7a419f761460907904721e93b66622f202088"}
Sep 10 15:42:51.137550 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.137507    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wnlqg" event={"ID":"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9","Type":"ContainerStarted","Data":"b5aa5f0f9d8fd99079aa3a046b931f8451bf0755ac3161905b1d55f7ce9f2eec"}
Sep 10 15:42:51.137695 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.137572    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wnlqg" event={"ID":"909c47d6-65aa-4f4f-bfc3-da05c15bc3c9","Type":"ContainerStarted","Data":"838ac8144c944d029388290eea1ab012c23018e9fb20effd186d6328efc00773"}
Sep 10 15:42:51.137695 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.137606    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:42:51.139703 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.139675    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j" event={"ID":"83488306-a407-4c7f-9309-d7d47955d038","Type":"ContainerStarted","Data":"8f89abd7202c9eae132e305e8c6ce4524b8c8fd72e008fecc5b27958a4122903"}
Sep 10 15:42:51.139826 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.139732    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j" event={"ID":"83488306-a407-4c7f-9309-d7d47955d038","Type":"ContainerStarted","Data":"40bfdcac655280feda646f85845f07579db4045aacc58fe4e9a750d4679106fe"}
Sep 10 15:42:51.145444 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.145388    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-tggpc" podStartSLOduration=57.051827715 podStartE2EDuration="1m1.145369755s" totalImagesPullingTime="4.09354204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:51.144010496 +0000 UTC m=+67.060135400" watchObservedRunningTime="2026-09-10 15:42:51.145369755 +0000 UTC m=+67.061494634"
Sep 10 15:42:51.160097 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.160034    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-d47zd" podStartSLOduration=43.226605607 podStartE2EDuration="47.160017547s" totalImagesPullingTime="3.93341194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:51.15900149 +0000 UTC m=+67.075126382" watchObservedRunningTime="2026-09-10 15:42:51.160017547 +0000 UTC m=+67.076142425"
Sep 10 15:42:51.173456 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.173380    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wnlqg" podStartSLOduration=34.142123293 podStartE2EDuration="38.173359136s" totalImagesPullingTime="4.031235843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:51.173234892 +0000 UTC m=+67.089359782" watchObservedRunningTime="2026-09-10 15:42:51.173359136 +0000 UTC m=+67.089484016"
Sep 10 15:42:51.189479 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:51.189422    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qw66j" podStartSLOduration=57.238884237 podStartE2EDuration="1m1.189406036s" totalImagesPullingTime="3.950521799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:51.18879442 +0000 UTC m=+67.104919313" watchObservedRunningTime="2026-09-10 15:42:51.189406036 +0000 UTC m=+67.105530915"
Sep 10 15:42:53.152420 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:53.152310    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5c6r7" event={"ID":"511e3fe0-d13f-413c-9877-1e056560de23","Type":"ContainerStarted","Data":"2824491c45cbdcc95635a7cbb561851b7ae56a5024c512d7ee05f010050c9b65"}
Sep 10 15:42:53.152420 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:53.152393    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5c6r7" event={"ID":"511e3fe0-d13f-413c-9877-1e056560de23","Type":"ContainerStarted","Data":"1ea03503b1f871bf3060d3d54114e4905e5b11a3ec0485ec2c5cdd2a12739018"}
Sep 10 15:42:53.154643 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:53.154613    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gqclh" event={"ID":"ee6b2a6f-4882-4cce-96cf-a8225b5b864e","Type":"ContainerStarted","Data":"9857783208a524d9150bab8e7e6a9f34c2743d5c76ae7f81ce5d997d28e6146e"}
Sep 10 15:42:53.169282 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:53.169218    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5c6r7" podStartSLOduration=66.403928461 podStartE2EDuration="1m8.169198834s" totalImagesPullingTime="1.765270373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:53.166968448 +0000 UTC m=+69.083093327" watchObservedRunningTime="2026-09-10 15:42:53.169198834 +0000 UTC m=+69.085323712"
Sep 10 15:42:55.085733 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.085694    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-v5z46"]
Sep 10 15:42:55.113181 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.113126    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"]
Sep 10 15:42:55.113370 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.113307    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-v5z46"
Sep 10 15:42:55.115944 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.115907    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 15:42:55.116460 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.116274    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-htfdg\""
Sep 10 15:42:55.116460 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.116306    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-v5z46"]
Sep 10 15:42:55.116460 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.116345    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"]
Sep 10 15:42:55.116460 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.116432    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"
Sep 10 15:42:55.116460 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.116454    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 15:42:55.119583 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.119552    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-p9ncn\""
Sep 10 15:42:55.119870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.119844    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 15:42:55.166334 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.166295    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gqclh" event={"ID":"ee6b2a6f-4882-4cce-96cf-a8225b5b864e","Type":"ContainerStarted","Data":"224f58377414c3bd0a330aabb257dd418e20f699d808cc5db036f8b131fab592"}
Sep 10 15:42:55.202371 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.202307    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gqclh" podStartSLOduration=20.703851584 podStartE2EDuration="24.202289277s" totalImagesPullingTime="3.498437693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:42:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:55.200759186 +0000 UTC m=+71.116884085" watchObservedRunningTime="2026-09-10 15:42:55.202289277 +0000 UTC m=+71.118414155"
Sep 10 15:42:55.245834 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.245780    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsh2b\" (UniqueName: \"kubernetes.io/projected/b6d5f4ab-9262-4c92-a259-a2ef772863bd-kube-api-access-rsh2b\") pod \"downloads-85d98c998f-v5z46\" (UID: \"b6d5f4ab-9262-4c92-a259-a2ef772863bd\") " pod="openshift-console/downloads-85d98c998f-v5z46"
Sep 10 15:42:55.246040 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.245884    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ebdbdd63-3263-4d90-8cec-8d2b22bdb693-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-lt4pw\" (UID: \"ebdbdd63-3263-4d90-8cec-8d2b22bdb693\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"
Sep 10 15:42:55.347479 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.347386    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsh2b\" (UniqueName: \"kubernetes.io/projected/b6d5f4ab-9262-4c92-a259-a2ef772863bd-kube-api-access-rsh2b\") pod \"downloads-85d98c998f-v5z46\" (UID: \"b6d5f4ab-9262-4c92-a259-a2ef772863bd\") " pod="openshift-console/downloads-85d98c998f-v5z46"
Sep 10 15:42:55.347631 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.347547    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ebdbdd63-3263-4d90-8cec-8d2b22bdb693-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-lt4pw\" (UID: \"ebdbdd63-3263-4d90-8cec-8d2b22bdb693\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"
Sep 10 15:42:55.350286 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.350259    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ebdbdd63-3263-4d90-8cec-8d2b22bdb693-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-lt4pw\" (UID: \"ebdbdd63-3263-4d90-8cec-8d2b22bdb693\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"
Sep 10 15:42:55.356837 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.356805    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsh2b\" (UniqueName: \"kubernetes.io/projected/b6d5f4ab-9262-4c92-a259-a2ef772863bd-kube-api-access-rsh2b\") pod \"downloads-85d98c998f-v5z46\" (UID: \"b6d5f4ab-9262-4c92-a259-a2ef772863bd\") " pod="openshift-console/downloads-85d98c998f-v5z46"
Sep 10 15:42:55.431670 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.431626    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-v5z46"
Sep 10 15:42:55.438766 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.438738    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"
Sep 10 15:42:55.655172 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.655082    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f5dc4bf7-kfh77"]
Sep 10 15:42:55.686544 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.686506    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f5dc4bf7-kfh77"]
Sep 10 15:42:55.686755 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.686566    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.691187 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.691163    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 15:42:55.691358 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.691341    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 15:42:55.691426 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.691406    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mkfbn\""
Sep 10 15:42:55.691554 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.691537    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 15:42:55.691802 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.691784    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 15:42:55.691875 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.691855    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 15:42:55.732345 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.732231    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-v5z46"]
Sep 10 15:42:55.735211 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:55.735176    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d5f4ab_9262_4c92_a259_a2ef772863bd.slice/crio-65ff32741687d40855817ff5fe13d597ce035ea43398cf0e0900121dbfd16e14 WatchSource:0}: Error finding container 65ff32741687d40855817ff5fe13d597ce035ea43398cf0e0900121dbfd16e14: Status 404 returned error can't find the container with id 65ff32741687d40855817ff5fe13d597ce035ea43398cf0e0900121dbfd16e14
Sep 10 15:42:55.759512 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.759476    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"]
Sep 10 15:42:55.763773 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:55.763746    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebdbdd63_3263_4d90_8cec_8d2b22bdb693.slice/crio-469f3a54c5ec5ca8bacf05f0474c5b0a71bc7be8571841429b5092ac4daffb25 WatchSource:0}: Error finding container 469f3a54c5ec5ca8bacf05f0474c5b0a71bc7be8571841429b5092ac4daffb25: Status 404 returned error can't find the container with id 469f3a54c5ec5ca8bacf05f0474c5b0a71bc7be8571841429b5092ac4daffb25
Sep 10 15:42:55.852585 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.852541    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-service-ca\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.852748 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.852654    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-serving-cert\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.852748 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.852685    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-oauth-serving-cert\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.852748 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.852723    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4zzd\" (UniqueName: \"kubernetes.io/projected/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-kube-api-access-d4zzd\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.852930 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.852789    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-config\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.852930 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.852846    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-oauth-config\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.954392 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.954281    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-service-ca\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.954392 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.954351    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-serving-cert\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.954392 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.954373    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-oauth-serving-cert\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.954392 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.954398    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4zzd\" (UniqueName: \"kubernetes.io/projected/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-kube-api-access-d4zzd\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.954724 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.954422    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-config\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.954724 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.954438    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-oauth-config\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.955089 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.955059    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-service-ca\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.955256 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.955224    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-oauth-serving-cert\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.955301 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.955224    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-config\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.957033 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.957017    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-oauth-config\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.957033 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.957027    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-serving-cert\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.962427 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.962407    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4zzd\" (UniqueName: \"kubernetes.io/projected/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-kube-api-access-d4zzd\") pod \"console-f5dc4bf7-kfh77\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") " pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:55.997171 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:55.997117    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:42:56.172116 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:56.172076    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw" event={"ID":"ebdbdd63-3263-4d90-8cec-8d2b22bdb693","Type":"ContainerStarted","Data":"469f3a54c5ec5ca8bacf05f0474c5b0a71bc7be8571841429b5092ac4daffb25"}
Sep 10 15:42:56.173402 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:56.173368    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-v5z46" event={"ID":"b6d5f4ab-9262-4c92-a259-a2ef772863bd","Type":"ContainerStarted","Data":"65ff32741687d40855817ff5fe13d597ce035ea43398cf0e0900121dbfd16e14"}
Sep 10 15:42:56.301059 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:56.301010    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f5dc4bf7-kfh77"]
Sep 10 15:42:56.304221 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:42:56.304178    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda50c9d_1aa5_4daf_96ec_fea5bcac0818.slice/crio-08834e5e5fe5e0e7a7c1714eb99c426779603d9513ba52c68af04d8628eae5a3 WatchSource:0}: Error finding container 08834e5e5fe5e0e7a7c1714eb99c426779603d9513ba52c68af04d8628eae5a3: Status 404 returned error can't find the container with id 08834e5e5fe5e0e7a7c1714eb99c426779603d9513ba52c68af04d8628eae5a3
Sep 10 15:42:57.178670 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:57.178633    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5dc4bf7-kfh77" event={"ID":"bda50c9d-1aa5-4daf-96ec-fea5bcac0818","Type":"ContainerStarted","Data":"08834e5e5fe5e0e7a7c1714eb99c426779603d9513ba52c68af04d8628eae5a3"}
Sep 10 15:42:58.186888 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:58.186847    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw" event={"ID":"ebdbdd63-3263-4d90-8cec-8d2b22bdb693","Type":"ContainerStarted","Data":"b3757d4f8ae4cc9e919dcdc089c68bef3df50ba454f9fd3b7c8645126b30a4d8"}
Sep 10 15:42:58.187450 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:58.187089    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"
Sep 10 15:42:58.193228 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:58.193196    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw"
Sep 10 15:42:58.228320 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:42:58.228252    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-lt4pw" podStartSLOduration=1.6539274800000001 podStartE2EDuration="3.228194308s" totalImagesPullingTime="1.574266828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:42:58.206648253 +0000 UTC m=+74.122773132" watchObservedRunningTime="2026-09-10 15:42:58.228194308 +0000 UTC m=+74.144319187"
Sep 10 15:43:01.148646 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:01.148613    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wnlqg"
Sep 10 15:43:01.205430 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:01.205390    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5dc4bf7-kfh77" event={"ID":"bda50c9d-1aa5-4daf-96ec-fea5bcac0818","Type":"ContainerStarted","Data":"9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930"}
Sep 10 15:43:01.226926 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:01.226862    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f5dc4bf7-kfh77" podStartSLOduration=1.7634745010000001 podStartE2EDuration="6.226837087s" totalImagesPullingTime="4.463362586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:43:01.22417506 +0000 UTC m=+77.140299968" watchObservedRunningTime="2026-09-10 15:43:01.226837087 +0000 UTC m=+77.142961967"
Sep 10 15:43:03.023077 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:03.023040    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qswkd"
Sep 10 15:43:04.682321 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.682287    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5z6dv"]
Sep 10 15:43:04.685823 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.685791    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.688721 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.688530    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 15:43:04.689647 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.689393    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 15:43:04.689647 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.689442    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 15:43:04.689647 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.689477    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 15:43:04.689647 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.689498    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j55xj\""
Sep 10 15:43:04.725246 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.725209    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-textfile\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.725464 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.725256    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-wtmp\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.725464 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.725291    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.725464 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.725367    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-root\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.725639 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.725437    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.725639 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.725587    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-sys\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.725639 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.725630    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-tls\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.725775 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.725652    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-metrics-client-ca\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.725775 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.725701    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4crn4\" (UniqueName: \"kubernetes.io/projected/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-kube-api-access-4crn4\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826350 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826312    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826558 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826377    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-sys\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826558 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826408    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-tls\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826558 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826433    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-metrics-client-ca\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826558 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826456    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4crn4\" (UniqueName: \"kubernetes.io/projected/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-kube-api-access-4crn4\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826558 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826517    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-sys\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826823 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826639    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-textfile\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826823 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826678    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-wtmp\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826823 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826767    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826964 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826868    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-root\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.826964 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.826955    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-root\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.827057 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.827040    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-textfile\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.827108 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.827055    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-metrics-client-ca\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.827108 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.827040    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-accelerators-collector-config\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.827108 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.827066    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-wtmp\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.829462 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.829420    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-tls\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.829602 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.829585    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:04.835793 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:04.835766    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4crn4\" (UniqueName: \"kubernetes.io/projected/5e19361d-b37c-46a6-97a2-24c2c26b3eb0-kube-api-access-4crn4\") pod \"node-exporter-5z6dv\" (UID: \"5e19361d-b37c-46a6-97a2-24c2c26b3eb0\") " pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:05.000713 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:05.000626    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5z6dv"
Sep 10 15:43:05.014955 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:43:05.014917    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e19361d_b37c_46a6_97a2_24c2c26b3eb0.slice/crio-62d8447ef3291cf3a7e536fadf6cb6577b582936a0a9c97a4ba11ea9419ee380 WatchSource:0}: Error finding container 62d8447ef3291cf3a7e536fadf6cb6577b582936a0a9c97a4ba11ea9419ee380: Status 404 returned error can't find the container with id 62d8447ef3291cf3a7e536fadf6cb6577b582936a0a9c97a4ba11ea9419ee380
Sep 10 15:43:05.226595 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:05.226549    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5z6dv" event={"ID":"5e19361d-b37c-46a6-97a2-24c2c26b3eb0","Type":"ContainerStarted","Data":"62d8447ef3291cf3a7e536fadf6cb6577b582936a0a9c97a4ba11ea9419ee380"}
Sep 10 15:43:05.998284 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:05.997476    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:43:05.998284 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:05.997570    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:43:05.999679 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:05.999567    2527 patch_prober.go:28] interesting pod/console-f5dc4bf7-kfh77 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.27:8443/health\": dial tcp 10.132.0.27:8443: connect: connection refused" start-of-body=
Sep 10 15:43:05.999679 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:05.999639    2527 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-f5dc4bf7-kfh77" podUID="bda50c9d-1aa5-4daf-96ec-fea5bcac0818" containerName="console" probeResult="failure" output="Get \"https://10.132.0.27:8443/health\": dial tcp 10.132.0.27:8443: connect: connection refused"
Sep 10 15:43:06.061207 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:06.061158    2527 patch_prober.go:28] interesting pod/image-registry-67ddd8b586-7pt28 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 15:43:06.061402 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:06.061257    2527 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28" podUID="abc19d51-c8e1-4f57-88c8-2004ec54c4bb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 15:43:07.239517 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:07.239396    2527 generic.go:356] "Generic (PLEG): container finished" podID="5e19361d-b37c-46a6-97a2-24c2c26b3eb0" containerID="91d01a8202bb3023fd0838c64510de76af334ae8d9d1c292e755628249e17c50" exitCode=0
Sep 10 15:43:07.239517 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:07.239464    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5z6dv" event={"ID":"5e19361d-b37c-46a6-97a2-24c2c26b3eb0","Type":"ContainerDied","Data":"91d01a8202bb3023fd0838c64510de76af334ae8d9d1c292e755628249e17c50"}
Sep 10 15:43:08.114865 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:08.114829    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:43:08.247705 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:08.247663    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5z6dv" event={"ID":"5e19361d-b37c-46a6-97a2-24c2c26b3eb0","Type":"ContainerStarted","Data":"40ce0cde5a81f4aa5210f54608d937337356fb596edc1b301367766e2f143479"}
Sep 10 15:43:08.248180 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:08.247733    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5z6dv" event={"ID":"5e19361d-b37c-46a6-97a2-24c2c26b3eb0","Type":"ContainerStarted","Data":"cf634db721b8d9a819ee00d2aa1d5aa0efbbbed4fa58124dad9c7fca08d75b2d"}
Sep 10 15:43:08.273519 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:08.273459    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5z6dv" podStartSLOduration=2.939544485 podStartE2EDuration="4.273433631s" totalImagesPullingTime="1.271163329s" totalInitContainerRuntime="62.725817ms" isStatefulPod=true podCreationTimestamp="2026-09-10 15:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:43:08.27170658 +0000 UTC m=+84.187831492" watchObservedRunningTime="2026-09-10 15:43:08.273433631 +0000 UTC m=+84.189558512"
Sep 10 15:43:08.645537 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:08.645499    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f5dc4bf7-kfh77"]
Sep 10 15:43:15.285191 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:15.285117    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-v5z46" event={"ID":"b6d5f4ab-9262-4c92-a259-a2ef772863bd","Type":"ContainerStarted","Data":"bd497fea0fe1dc2ce2464cdc51d7fcfc517b4bb93ba27be9a2f3ee5a7f3f3b8c"}
Sep 10 15:43:15.285672 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:15.285280    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-v5z46"
Sep 10 15:43:15.299482 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:15.299440    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-v5z46"
Sep 10 15:43:15.306079 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:15.306007    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-v5z46" podStartSLOduration=1.350526632 podStartE2EDuration="20.305983271s" totalImagesPullingTime="18.955456639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:42:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:43:15.304126186 +0000 UTC m=+91.220251080" watchObservedRunningTime="2026-09-10 15:43:15.305983271 +0000 UTC m=+91.222108156"
Sep 10 15:43:18.355690 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:18.355646    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67ddd8b586-7pt28"]
Sep 10 15:43:33.674413 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:33.674346    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f5dc4bf7-kfh77" podUID="bda50c9d-1aa5-4daf-96ec-fea5bcac0818" containerName="console" containerID="cri-o://9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930" gracePeriod=15
Sep 10 15:43:33.974407 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:33.974363    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:43:34.099203 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.099161    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-service-ca\") pod \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") "
Sep 10 15:43:34.099203 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.099201    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-config\") pod \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") "
Sep 10 15:43:34.099442 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.099251    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-serving-cert\") pod \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") "
Sep 10 15:43:34.099442 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.099297    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-oauth-config\") pod \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") "
Sep 10 15:43:34.099442 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.099322    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4zzd\" (UniqueName: \"kubernetes.io/projected/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-kube-api-access-d4zzd\") pod \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") "
Sep 10 15:43:34.099442 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.099393    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-oauth-serving-cert\") pod \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\" (UID: \"bda50c9d-1aa5-4daf-96ec-fea5bcac0818\") "
Sep 10 15:43:34.099721 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.099686    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-service-ca" (OuterVolumeSpecName: "service-ca") pod "bda50c9d-1aa5-4daf-96ec-fea5bcac0818" (UID: "bda50c9d-1aa5-4daf-96ec-fea5bcac0818"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:34.099721 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.099710    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-config" (OuterVolumeSpecName: "console-config") pod "bda50c9d-1aa5-4daf-96ec-fea5bcac0818" (UID: "bda50c9d-1aa5-4daf-96ec-fea5bcac0818"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:34.099849 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.099732    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bda50c9d-1aa5-4daf-96ec-fea5bcac0818" (UID: "bda50c9d-1aa5-4daf-96ec-fea5bcac0818"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:34.101736 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.101713    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bda50c9d-1aa5-4daf-96ec-fea5bcac0818" (UID: "bda50c9d-1aa5-4daf-96ec-fea5bcac0818"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:43:34.101814 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.101777    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-kube-api-access-d4zzd" (OuterVolumeSpecName: "kube-api-access-d4zzd") pod "bda50c9d-1aa5-4daf-96ec-fea5bcac0818" (UID: "bda50c9d-1aa5-4daf-96ec-fea5bcac0818"). InnerVolumeSpecName "kube-api-access-d4zzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:34.101850 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.101823    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bda50c9d-1aa5-4daf-96ec-fea5bcac0818" (UID: "bda50c9d-1aa5-4daf-96ec-fea5bcac0818"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:43:34.200216 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.200094    2527 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-oauth-config\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:34.200216 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.200129    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4zzd\" (UniqueName: \"kubernetes.io/projected/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-kube-api-access-d4zzd\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:34.200216 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.200165    2527 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-oauth-serving-cert\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:34.200216 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.200174    2527 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:34.200216 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.200182    2527 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-config\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:34.200216 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.200189    2527 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda50c9d-1aa5-4daf-96ec-fea5bcac0818-console-serving-cert\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:34.375554 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.375513    2527 generic.go:356] "Generic (PLEG): container finished" podID="bda50c9d-1aa5-4daf-96ec-fea5bcac0818" containerID="9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930" exitCode=0
Sep 10 15:43:34.375697 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.375558    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f5dc4bf7-kfh77"
Sep 10 15:43:34.375697 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.375596    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5dc4bf7-kfh77" event={"ID":"bda50c9d-1aa5-4daf-96ec-fea5bcac0818","Type":"ContainerDied","Data":"9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930"}
Sep 10 15:43:34.375697 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.375653    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f5dc4bf7-kfh77" event={"ID":"bda50c9d-1aa5-4daf-96ec-fea5bcac0818","Type":"ContainerDied","Data":"08834e5e5fe5e0e7a7c1714eb99c426779603d9513ba52c68af04d8628eae5a3"}
Sep 10 15:43:34.375697 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.375679    2527 scope.go:117] "RemoveContainer" containerID="9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930"
Sep 10 15:43:34.385366 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.385341    2527 scope.go:117] "RemoveContainer" containerID="9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930"
Sep 10 15:43:34.385693 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:43:34.385669    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930\": container with ID starting with 9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930 not found: ID does not exist" containerID="9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930"
Sep 10 15:43:34.385744 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.385704    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930"} err="failed to get container status \"9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930\": rpc error: code = NotFound desc = could not find container \"9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930\": container with ID starting with 9642b621e8e0d5a47ca8eb21677458a1b9eb7f6ae64d46898ccbb5038d4af930 not found: ID does not exist"
Sep 10 15:43:34.397954 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.397922    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f5dc4bf7-kfh77"]
Sep 10 15:43:34.401615 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.401585    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f5dc4bf7-kfh77"]
Sep 10 15:43:34.563735 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:34.563694    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bda50c9d-1aa5-4daf-96ec-fea5bcac0818" path="/var/lib/kubelet/pods/bda50c9d-1aa5-4daf-96ec-fea5bcac0818/volumes"
Sep 10 15:43:43.380658 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.380586    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28" podUID="abc19d51-c8e1-4f57-88c8-2004ec54c4bb" containerName="registry" containerID="cri-o://0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54" gracePeriod=30
Sep 10 15:43:43.666451 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.666413    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:43:43.788615 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.788571    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-ca-trust-extracted\") pod \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") "
Sep 10 15:43:43.788816 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.788642    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") pod \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") "
Sep 10 15:43:43.788816 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.788669    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-image-registry-private-configuration\") pod \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") "
Sep 10 15:43:43.788816 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.788686    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-trusted-ca\") pod \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") "
Sep 10 15:43:43.788816 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.788706    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-installation-pull-secrets\") pod \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") "
Sep 10 15:43:43.788816 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.788756    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7q8v\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-kube-api-access-s7q8v\") pod \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") "
Sep 10 15:43:43.789067 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.788852    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-bound-sa-token\") pod \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") "
Sep 10 15:43:43.789067 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.788937    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-certificates\") pod \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\" (UID: \"abc19d51-c8e1-4f57-88c8-2004ec54c4bb\") "
Sep 10 15:43:43.789327 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.789298    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "abc19d51-c8e1-4f57-88c8-2004ec54c4bb" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:43.789706 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.789645    2527 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-trusted-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:43.789869 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.789842    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "abc19d51-c8e1-4f57-88c8-2004ec54c4bb" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:43:43.791543 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.791486    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "abc19d51-c8e1-4f57-88c8-2004ec54c4bb" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:43.791543 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.791500    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "abc19d51-c8e1-4f57-88c8-2004ec54c4bb" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:43:43.791543 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.791526    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "abc19d51-c8e1-4f57-88c8-2004ec54c4bb" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:43.791748 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.791620    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "abc19d51-c8e1-4f57-88c8-2004ec54c4bb" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:43:43.791748 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.791650    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-kube-api-access-s7q8v" (OuterVolumeSpecName: "kube-api-access-s7q8v") pod "abc19d51-c8e1-4f57-88c8-2004ec54c4bb" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb"). InnerVolumeSpecName "kube-api-access-s7q8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:43:43.797762 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.797734    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "abc19d51-c8e1-4f57-88c8-2004ec54c4bb" (UID: "abc19d51-c8e1-4f57-88c8-2004ec54c4bb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:43:43.890498 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.890412    2527 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-certificates\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:43.890498 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.890444    2527 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-ca-trust-extracted\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:43.890498 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.890454    2527 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-registry-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:43.890498 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.890463    2527 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-image-registry-private-configuration\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:43.890498 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.890472    2527 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-installation-pull-secrets\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:43.890498 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.890482    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7q8v\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-kube-api-access-s7q8v\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:43.890498 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:43.890490    2527 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abc19d51-c8e1-4f57-88c8-2004ec54c4bb-bound-sa-token\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:43:44.422044 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.422008    2527 generic.go:356] "Generic (PLEG): container finished" podID="abc19d51-c8e1-4f57-88c8-2004ec54c4bb" containerID="0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54" exitCode=0
Sep 10 15:43:44.422535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.422052    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28"
Sep 10 15:43:44.422535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.422059    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28" event={"ID":"abc19d51-c8e1-4f57-88c8-2004ec54c4bb","Type":"ContainerDied","Data":"0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54"}
Sep 10 15:43:44.422535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.422102    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67ddd8b586-7pt28" event={"ID":"abc19d51-c8e1-4f57-88c8-2004ec54c4bb","Type":"ContainerDied","Data":"edd1d8bbf7bf57cc1db73b851d9244b04d90597310a2649777708712ee7e79db"}
Sep 10 15:43:44.422535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.422122    2527 scope.go:117] "RemoveContainer" containerID="0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54"
Sep 10 15:43:44.431321 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.431298    2527 scope.go:117] "RemoveContainer" containerID="0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54"
Sep 10 15:43:44.431696 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:43:44.431676    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54\": container with ID starting with 0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54 not found: ID does not exist" containerID="0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54"
Sep 10 15:43:44.431745 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.431709    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54"} err="failed to get container status \"0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54\": rpc error: code = NotFound desc = could not find container \"0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54\": container with ID starting with 0b972026c477c83a38e491399768a1fcb0998fb6512a77226314206343a8eb54 not found: ID does not exist"
Sep 10 15:43:44.443289 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.443258    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67ddd8b586-7pt28"]
Sep 10 15:43:44.448657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.448624    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67ddd8b586-7pt28"]
Sep 10 15:43:44.562992 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:44.562940    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abc19d51-c8e1-4f57-88c8-2004ec54c4bb" path="/var/lib/kubelet/pods/abc19d51-c8e1-4f57-88c8-2004ec54c4bb/volumes"
Sep 10 15:43:47.435913 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:47.435874    2527 generic.go:356] "Generic (PLEG): container finished" podID="8101fd3c-1dcc-4099-88af-6581442ee257" containerID="30ce2c209f4b1e256702c08ec5354fd92a32f5d7c194a0b931b3f7960345a948" exitCode=0
Sep 10 15:43:47.436408 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:47.435948    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t" event={"ID":"8101fd3c-1dcc-4099-88af-6581442ee257","Type":"ContainerDied","Data":"30ce2c209f4b1e256702c08ec5354fd92a32f5d7c194a0b931b3f7960345a948"}
Sep 10 15:43:47.436408 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:47.436343    2527 scope.go:117] "RemoveContainer" containerID="30ce2c209f4b1e256702c08ec5354fd92a32f5d7c194a0b931b3f7960345a948"
Sep 10 15:43:48.441996 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:48.441959    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-5gl8t" event={"ID":"8101fd3c-1dcc-4099-88af-6581442ee257","Type":"ContainerStarted","Data":"a5d0e90fa52d1455a0c9b246916102fac5c0c233d25bf7a284b0129b3ab3fc9b"}
Sep 10 15:43:51.454983 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:51.454893    2527 generic.go:356] "Generic (PLEG): container finished" podID="2b7b241e-708b-40bb-ba3d-52c0fee5e425" containerID="aa75eb5aa0be8f34c91c71aae5140b4cfaa971d018b66263cadc0b70970257d5" exitCode=0
Sep 10 15:43:51.454983 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:51.454955    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4" event={"ID":"2b7b241e-708b-40bb-ba3d-52c0fee5e425","Type":"ContainerDied","Data":"aa75eb5aa0be8f34c91c71aae5140b4cfaa971d018b66263cadc0b70970257d5"}
Sep 10 15:43:51.455435 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:51.455289    2527 scope.go:117] "RemoveContainer" containerID="aa75eb5aa0be8f34c91c71aae5140b4cfaa971d018b66263cadc0b70970257d5"
Sep 10 15:43:52.460634 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:43:52.460600    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7bdj4" event={"ID":"2b7b241e-708b-40bb-ba3d-52c0fee5e425","Type":"ContainerStarted","Data":"4bc3e71634d37038aba24bbb58105e79a2a8efb51f69bfa46e76a5b0316154b9"}
Sep 10 15:44:02.506530 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:02.506494    2527 generic.go:356] "Generic (PLEG): container finished" podID="ee34d318-9263-4aeb-a95f-316447f94aa2" containerID="b149a36577c66c5ca7fe7c18184a693972cf716fd717dd59ded92ca382b41e55" exitCode=0
Sep 10 15:44:02.507025 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:02.506565    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc" event={"ID":"ee34d318-9263-4aeb-a95f-316447f94aa2","Type":"ContainerDied","Data":"b149a36577c66c5ca7fe7c18184a693972cf716fd717dd59ded92ca382b41e55"}
Sep 10 15:44:02.507025 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:02.506957    2527 scope.go:117] "RemoveContainer" containerID="b149a36577c66c5ca7fe7c18184a693972cf716fd717dd59ded92ca382b41e55"
Sep 10 15:44:03.512864 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:03.512829    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-fjlvc" event={"ID":"ee34d318-9263-4aeb-a95f-316447f94aa2","Type":"ContainerStarted","Data":"814d4060e8196f4e65b4858ea14cc4a14333e228ddabaadbccbf49cf9055091e"}
Sep 10 15:44:04.503690 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:04.503646    2527 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" podUID="67f934b5-d4b9-49af-a439-0e77d6890343" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 10 15:44:14.503415 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:14.503370    2527 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" podUID="67f934b5-d4b9-49af-a439-0e77d6890343" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 10 15:44:24.504186 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:24.504126    2527 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" podUID="67f934b5-d4b9-49af-a439-0e77d6890343" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 10 15:44:24.504663 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:24.504213    2527 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q"
Sep 10 15:44:24.504720 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:24.504703    2527 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"7332099e25ad94ee8130edd0aebf85531c90b3b47f816af550fb2ae9bb51de17"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 10 15:44:24.504756 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:24.504739    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" podUID="67f934b5-d4b9-49af-a439-0e77d6890343" containerName="service-proxy" containerID="cri-o://7332099e25ad94ee8130edd0aebf85531c90b3b47f816af550fb2ae9bb51de17" gracePeriod=30
Sep 10 15:44:25.605786 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:25.605754    2527 generic.go:356] "Generic (PLEG): container finished" podID="67f934b5-d4b9-49af-a439-0e77d6890343" containerID="7332099e25ad94ee8130edd0aebf85531c90b3b47f816af550fb2ae9bb51de17" exitCode=2
Sep 10 15:44:25.606193 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:25.605835    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" event={"ID":"67f934b5-d4b9-49af-a439-0e77d6890343","Type":"ContainerDied","Data":"7332099e25ad94ee8130edd0aebf85531c90b3b47f816af550fb2ae9bb51de17"}
Sep 10 15:44:25.606193 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:44:25.605875    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-868d69f696-nmb5q" event={"ID":"67f934b5-d4b9-49af-a439-0e77d6890343","Type":"ContainerStarted","Data":"e632c29b68bdefd67cf8a678e3099705b45d132c12c9f9decf5555151ae7cb01"}
Sep 10 15:46:44.486515 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:46:44.486477    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 15:46:44.487065 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:46:44.487040    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 15:46:44.496513 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:46:44.496487    2527 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 15:47:29.936458 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.936414    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"]
Sep 10 15:47:29.937049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.936869    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abc19d51-c8e1-4f57-88c8-2004ec54c4bb" containerName="registry"
Sep 10 15:47:29.937049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.936889    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="abc19d51-c8e1-4f57-88c8-2004ec54c4bb" containerName="registry"
Sep 10 15:47:29.937049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.936908    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bda50c9d-1aa5-4daf-96ec-fea5bcac0818" containerName="console"
Sep 10 15:47:29.937049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.936917    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda50c9d-1aa5-4daf-96ec-fea5bcac0818" containerName="console"
Sep 10 15:47:29.937049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.937018    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="abc19d51-c8e1-4f57-88c8-2004ec54c4bb" containerName="registry"
Sep 10 15:47:29.937049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.937035    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="bda50c9d-1aa5-4daf-96ec-fea5bcac0818" containerName="console"
Sep 10 15:47:29.940150 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.940111    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:29.943283 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.943259    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 15:47:29.943415 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.943371    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 15:47:29.943724 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.943704    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 15:47:29.943814 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.943746    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wwh6s\""
Sep 10 15:47:29.943814 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.943780    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 15:47:29.967455 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:29.967413    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"]
Sep 10 15:47:30.030395 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.030352    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.030395 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.030400    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gtkw\" (UniqueName: \"kubernetes.io/projected/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-kube-api-access-9gtkw\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.030679 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.030424    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-webhook-cert\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.030679 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.030498    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-manifests\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.030679 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.030533    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-tmp\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.131582 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.131539    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-manifests\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.131819 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.131599    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-tmp\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.131819 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.131655    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.131819 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.131689    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gtkw\" (UniqueName: \"kubernetes.io/projected/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-kube-api-access-9gtkw\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.131819 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.131720    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-webhook-cert\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.132065 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.132030    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-manifests\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.134250 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.134225    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-tmp\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.134536 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.134513    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.134586 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.134543    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-webhook-cert\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.142667 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.142639    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gtkw\" (UniqueName: \"kubernetes.io/projected/4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe-kube-api-access-9gtkw\") pod \"opendatahub-operator-controller-manager-97f7948ff-z25b9\" (UID: \"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe\") " pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.251720 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.251615    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:30.535020 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.534967    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"]
Sep 10 15:47:30.537952 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:47:30.537922    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb8cc7f_5a04_4be0_9b7d_708e8423a6fe.slice/crio-f2b873b5b7b214263cd82a41911d9159632cb2657c0570becbf3065acca8f605 WatchSource:0}: Error finding container f2b873b5b7b214263cd82a41911d9159632cb2657c0570becbf3065acca8f605: Status 404 returned error can't find the container with id f2b873b5b7b214263cd82a41911d9159632cb2657c0570becbf3065acca8f605
Sep 10 15:47:30.540330 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:30.540308    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:47:31.383796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:31.383710    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9" event={"ID":"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe","Type":"ContainerStarted","Data":"f2b873b5b7b214263cd82a41911d9159632cb2657c0570becbf3065acca8f605"}
Sep 10 15:47:34.402213 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:34.402165    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9" event={"ID":"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe","Type":"ContainerStarted","Data":"ca9569ac7fd12d3dc4aaa668ba50055bfe041d2ebfeb191961c1742ff3593922"}
Sep 10 15:47:35.408694 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:35.408657    2527 generic.go:356] "Generic (PLEG): container finished" podID="4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe" containerID="ca9569ac7fd12d3dc4aaa668ba50055bfe041d2ebfeb191961c1742ff3593922" exitCode=0
Sep 10 15:47:35.409210 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:35.408735    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9" event={"ID":"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe","Type":"ContainerDied","Data":"ca9569ac7fd12d3dc4aaa668ba50055bfe041d2ebfeb191961c1742ff3593922"}
Sep 10 15:47:36.415331 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:36.415236    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9" event={"ID":"4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe","Type":"ContainerStarted","Data":"4e005de51539694195423bf3c4161ce80b040d12d73684a71664f90fbfc09bf9"}
Sep 10 15:47:36.415803 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:36.415382    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:47:36.438921 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:36.438855    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9" podStartSLOduration=2.775623734 podStartE2EDuration="7.438830029s" totalImagesPullingTime="3.528826097s" totalInitContainerRuntime="1.134380198s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:47:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:47:36.437373977 +0000 UTC m=+352.353498868" watchObservedRunningTime="2026-09-10 15:47:36.438830029 +0000 UTC m=+352.354954912"
Sep 10 15:47:47.427427 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:47:47.427393    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-97f7948ff-z25b9"
Sep 10 15:48:08.521170 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.521104    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"]
Sep 10 15:48:08.524306 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.524287    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.528079 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.528053    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 15:48:08.530362 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.530332    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 15:48:08.532293 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.532266    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 15:48:08.532465 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.532293    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tbqck\""
Sep 10 15:48:08.532465 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.532314    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 15:48:08.532780 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.532765    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 15:48:08.544144 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.544097    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"]
Sep 10 15:48:08.590616 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.590578    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/67657edb-0b99-4240-b865-6dd8a955ed13-manager-config\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.590616 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.590617    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fzt2\" (UniqueName: \"kubernetes.io/projected/67657edb-0b99-4240-b865-6dd8a955ed13-kube-api-access-8fzt2\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.590841 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.590656    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67657edb-0b99-4240-b865-6dd8a955ed13-cert\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.590841 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.590725    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/67657edb-0b99-4240-b865-6dd8a955ed13-metrics-cert\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.692079 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.692042    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/67657edb-0b99-4240-b865-6dd8a955ed13-manager-config\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.692079 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.692081    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fzt2\" (UniqueName: \"kubernetes.io/projected/67657edb-0b99-4240-b865-6dd8a955ed13-kube-api-access-8fzt2\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.692335 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.692168    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67657edb-0b99-4240-b865-6dd8a955ed13-cert\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.692335 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.692203    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/67657edb-0b99-4240-b865-6dd8a955ed13-metrics-cert\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.692857 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.692832    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/67657edb-0b99-4240-b865-6dd8a955ed13-manager-config\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.694796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.694775    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/67657edb-0b99-4240-b865-6dd8a955ed13-metrics-cert\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.694909 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.694840    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67657edb-0b99-4240-b865-6dd8a955ed13-cert\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.700435 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.700408    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fzt2\" (UniqueName: \"kubernetes.io/projected/67657edb-0b99-4240-b865-6dd8a955ed13-kube-api-access-8fzt2\") pod \"lws-controller-manager-79dbb968b5-m588m\" (UID: \"67657edb-0b99-4240-b865-6dd8a955ed13\") " pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:08.834532 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:08.834491    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:09.136960 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:09.136874    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"]
Sep 10 15:48:09.141664 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:48:09.141632    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67657edb_0b99_4240_b865_6dd8a955ed13.slice/crio-a77ca49206934fd4a305de4199311888a64bd02d7aede961331a38e44f6bd4fe WatchSource:0}: Error finding container a77ca49206934fd4a305de4199311888a64bd02d7aede961331a38e44f6bd4fe: Status 404 returned error can't find the container with id a77ca49206934fd4a305de4199311888a64bd02d7aede961331a38e44f6bd4fe
Sep 10 15:48:09.573464 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:09.573420    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m" event={"ID":"67657edb-0b99-4240-b865-6dd8a955ed13","Type":"ContainerStarted","Data":"a77ca49206934fd4a305de4199311888a64bd02d7aede961331a38e44f6bd4fe"}
Sep 10 15:48:11.588993 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:11.588888    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m" event={"ID":"67657edb-0b99-4240-b865-6dd8a955ed13","Type":"ContainerStarted","Data":"49d255e15ce547afc6386c698a6d291cb3d643255a57f1c26afa678b724e78ee"}
Sep 10 15:48:11.589499 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:11.589078    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:48:11.610187 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:11.610102    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m" podStartSLOduration=1.416301851 podStartE2EDuration="3.610079511s" totalImagesPullingTime="2.19377766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:48:11.608207413 +0000 UTC m=+387.524332321" watchObservedRunningTime="2026-09-10 15:48:11.610079511 +0000 UTC m=+387.526204393"
Sep 10 15:48:22.595986 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:48:22.595952    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-79dbb968b5-m588m"
Sep 10 15:49:20.421683 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.421641    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"]
Sep 10 15:49:20.429371 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.429334    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:20.432871 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.432845    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 15:49:20.433043 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.432956    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5ls97\""
Sep 10 15:49:20.433353 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.433339    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 15:49:20.442607 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.442580    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"]
Sep 10 15:49:20.511689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.511640    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a912f526-ee89-4979-b6f7-26767d95858b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-klskt\" (UID: \"a912f526-ee89-4979-b6f7-26767d95858b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:20.511689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.511683    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8czmp\" (UniqueName: \"kubernetes.io/projected/a912f526-ee89-4979-b6f7-26767d95858b-kube-api-access-8czmp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-klskt\" (UID: \"a912f526-ee89-4979-b6f7-26767d95858b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:20.612699 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.612656    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a912f526-ee89-4979-b6f7-26767d95858b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-klskt\" (UID: \"a912f526-ee89-4979-b6f7-26767d95858b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:20.612699 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.612697    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8czmp\" (UniqueName: \"kubernetes.io/projected/a912f526-ee89-4979-b6f7-26767d95858b-kube-api-access-8czmp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-klskt\" (UID: \"a912f526-ee89-4979-b6f7-26767d95858b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:20.613064 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.613044    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a912f526-ee89-4979-b6f7-26767d95858b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-klskt\" (UID: \"a912f526-ee89-4979-b6f7-26767d95858b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:20.623734 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.623704    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8czmp\" (UniqueName: \"kubernetes.io/projected/a912f526-ee89-4979-b6f7-26767d95858b-kube-api-access-8czmp\") pod \"kuadrant-operator-controller-manager-76f8f686bf-klskt\" (UID: \"a912f526-ee89-4979-b6f7-26767d95858b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:20.740613 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:20.740525    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:21.032096 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:21.032055    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"]
Sep 10 15:49:21.036725 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:49:21.036691    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda912f526_ee89_4979_b6f7_26767d95858b.slice/crio-6175f85fad02a0e66a241371fdb24157fe19f93fa9207fb6f59842a76a0c3a83 WatchSource:0}: Error finding container 6175f85fad02a0e66a241371fdb24157fe19f93fa9207fb6f59842a76a0c3a83: Status 404 returned error can't find the container with id 6175f85fad02a0e66a241371fdb24157fe19f93fa9207fb6f59842a76a0c3a83
Sep 10 15:49:21.927965 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:21.927894    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt" event={"ID":"a912f526-ee89-4979-b6f7-26767d95858b","Type":"ContainerStarted","Data":"6175f85fad02a0e66a241371fdb24157fe19f93fa9207fb6f59842a76a0c3a83"}
Sep 10 15:49:22.196048 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:22.195948    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"]
Sep 10 15:49:22.199996 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:22.199963    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"
Sep 10 15:49:22.202948 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:22.202797    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4lbqw\""
Sep 10 15:49:22.202948 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:22.202902    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 15:49:22.213848 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:22.213792    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"]
Sep 10 15:49:22.327966 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:22.327933    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6ww5\" (UniqueName: \"kubernetes.io/projected/d1e5ed87-b513-4c07-8597-1d4170632bcc-kube-api-access-l6ww5\") pod \"dns-operator-controller-manager-7b84cd7c9c-79974\" (UID: \"d1e5ed87-b513-4c07-8597-1d4170632bcc\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"
Sep 10 15:49:22.429147 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:22.429076    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6ww5\" (UniqueName: \"kubernetes.io/projected/d1e5ed87-b513-4c07-8597-1d4170632bcc-kube-api-access-l6ww5\") pod \"dns-operator-controller-manager-7b84cd7c9c-79974\" (UID: \"d1e5ed87-b513-4c07-8597-1d4170632bcc\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"
Sep 10 15:49:22.460365 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:22.460294    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6ww5\" (UniqueName: \"kubernetes.io/projected/d1e5ed87-b513-4c07-8597-1d4170632bcc-kube-api-access-l6ww5\") pod \"dns-operator-controller-manager-7b84cd7c9c-79974\" (UID: \"d1e5ed87-b513-4c07-8597-1d4170632bcc\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"
Sep 10 15:49:22.516967 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:22.516927    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"
Sep 10 15:49:23.503142 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:23.503099    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"]
Sep 10 15:49:23.506322 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:49:23.506275    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e5ed87_b513_4c07_8597_1d4170632bcc.slice/crio-4133b253c74b27eb57247a3b0cb9fac89b48b00981da0d6d977b7f1092b97d81 WatchSource:0}: Error finding container 4133b253c74b27eb57247a3b0cb9fac89b48b00981da0d6d977b7f1092b97d81: Status 404 returned error can't find the container with id 4133b253c74b27eb57247a3b0cb9fac89b48b00981da0d6d977b7f1092b97d81
Sep 10 15:49:23.942918 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:23.942874    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974" event={"ID":"d1e5ed87-b513-4c07-8597-1d4170632bcc","Type":"ContainerStarted","Data":"4133b253c74b27eb57247a3b0cb9fac89b48b00981da0d6d977b7f1092b97d81"}
Sep 10 15:49:24.951016 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:24.950973    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt" event={"ID":"a912f526-ee89-4979-b6f7-26767d95858b","Type":"ContainerStarted","Data":"7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98"}
Sep 10 15:49:24.951486 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:24.951085    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:24.978014 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:24.977955    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt" podStartSLOduration=1.163078703 podStartE2EDuration="4.977940502s" totalImagesPullingTime="3.814861799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:49:24.976836175 +0000 UTC m=+460.892961073" watchObservedRunningTime="2026-09-10 15:49:24.977940502 +0000 UTC m=+460.894065381"
Sep 10 15:49:25.957644 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:25.957593    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974" event={"ID":"d1e5ed87-b513-4c07-8597-1d4170632bcc","Type":"ContainerStarted","Data":"7f3c930b3d2be49aeb870e3cdcd8ed77c916aadf4f54e17692019734b637a72f"}
Sep 10 15:49:25.958166 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:25.957961    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"
Sep 10 15:49:26.014457 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:26.014349    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974" podStartSLOduration=1.792419822 podStartE2EDuration="4.014328888s" totalImagesPullingTime="2.221909066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:49:26.01318208 +0000 UTC m=+461.929306984" watchObservedRunningTime="2026-09-10 15:49:26.014328888 +0000 UTC m=+461.930453768"
Sep 10 15:49:27.314240 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.314203    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-t6nlf"]
Sep 10 15:49:27.317689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.317670    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf"
Sep 10 15:49:27.320360 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.320338    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8jg62\""
Sep 10 15:49:27.336268 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.336237    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-t6nlf"]
Sep 10 15:49:27.475531 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.475490    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7ctf\" (UniqueName: \"kubernetes.io/projected/d3f96f25-58fd-4b6a-87e6-1c848688cafb-kube-api-access-n7ctf\") pod \"authorino-operator-76b478bd78-t6nlf\" (UID: \"d3f96f25-58fd-4b6a-87e6-1c848688cafb\") " pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf"
Sep 10 15:49:27.576527 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.576433    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7ctf\" (UniqueName: \"kubernetes.io/projected/d3f96f25-58fd-4b6a-87e6-1c848688cafb-kube-api-access-n7ctf\") pod \"authorino-operator-76b478bd78-t6nlf\" (UID: \"d3f96f25-58fd-4b6a-87e6-1c848688cafb\") " pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf"
Sep 10 15:49:27.598474 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.598441    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7ctf\" (UniqueName: \"kubernetes.io/projected/d3f96f25-58fd-4b6a-87e6-1c848688cafb-kube-api-access-n7ctf\") pod \"authorino-operator-76b478bd78-t6nlf\" (UID: \"d3f96f25-58fd-4b6a-87e6-1c848688cafb\") " pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf"
Sep 10 15:49:27.628539 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.628501    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf"
Sep 10 15:49:27.915238 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.915196    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-t6nlf"]
Sep 10 15:49:27.917558 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:49:27.917520    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f96f25_58fd_4b6a_87e6_1c848688cafb.slice/crio-44e9d5fa9ae6c474cf34db7cceb43c87935f48e8c6aa863a77d24a483ffc4bf0 WatchSource:0}: Error finding container 44e9d5fa9ae6c474cf34db7cceb43c87935f48e8c6aa863a77d24a483ffc4bf0: Status 404 returned error can't find the container with id 44e9d5fa9ae6c474cf34db7cceb43c87935f48e8c6aa863a77d24a483ffc4bf0
Sep 10 15:49:27.970034 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:27.969998    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf" event={"ID":"d3f96f25-58fd-4b6a-87e6-1c848688cafb","Type":"ContainerStarted","Data":"44e9d5fa9ae6c474cf34db7cceb43c87935f48e8c6aa863a77d24a483ffc4bf0"}
Sep 10 15:49:29.981796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:29.981755    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf" event={"ID":"d3f96f25-58fd-4b6a-87e6-1c848688cafb","Type":"ContainerStarted","Data":"9e18183417f67770e339bf406b3eddeb1271f777b8aed4f4f50c418a23862927"}
Sep 10 15:49:29.982238 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:29.981857    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf"
Sep 10 15:49:30.016904 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:30.016845    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf" podStartSLOduration=1.8942439370000002 podStartE2EDuration="3.016828352s" totalImagesPullingTime="1.122584415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:49:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:49:30.015806442 +0000 UTC m=+465.931931325" watchObservedRunningTime="2026-09-10 15:49:30.016828352 +0000 UTC m=+465.932953230"
Sep 10 15:49:35.959292 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:35.959258    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:36.966188 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:36.966158    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-79974"
Sep 10 15:49:38.362812 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.362772    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"]
Sep 10 15:49:38.363235 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.362979    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt" podUID="a912f526-ee89-4979-b6f7-26767d95858b" containerName="manager" containerID="cri-o://7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98" gracePeriod=2
Sep 10 15:49:38.375398 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.375362    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"]
Sep 10 15:49:38.404116 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.404085    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"]
Sep 10 15:49:38.404504 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.404487    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a912f526-ee89-4979-b6f7-26767d95858b" containerName="manager"
Sep 10 15:49:38.404598 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.404507    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="a912f526-ee89-4979-b6f7-26767d95858b" containerName="manager"
Sep 10 15:49:38.404653 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.404611    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="a912f526-ee89-4979-b6f7-26767d95858b" containerName="manager"
Sep 10 15:49:38.407651 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.407629    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:38.410485 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:49:38.410444    2527 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-klskt\" is forbidden: User \"system:node:ip-10-0-136-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-61.ec2.internal' and this object" podUID="a912f526-ee89-4979-b6f7-26767d95858b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:38.437296 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.437214    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"]
Sep 10 15:49:38.470331 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.470297    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gbbnr\" (UID: \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:38.470490 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.470423    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs2rx\" (UniqueName: \"kubernetes.io/projected/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-kube-api-access-zs2rx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gbbnr\" (UID: \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:38.571314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.571258    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs2rx\" (UniqueName: \"kubernetes.io/projected/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-kube-api-access-zs2rx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gbbnr\" (UID: \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:38.571508 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.571389    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gbbnr\" (UID: \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:38.571775 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.571757    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gbbnr\" (UID: \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:38.584625 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.584591    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs2rx\" (UniqueName: \"kubernetes.io/projected/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-kube-api-access-zs2rx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gbbnr\" (UID: \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:38.620513 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.620448    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:38.672264 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.672222    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8czmp\" (UniqueName: \"kubernetes.io/projected/a912f526-ee89-4979-b6f7-26767d95858b-kube-api-access-8czmp\") pod \"a912f526-ee89-4979-b6f7-26767d95858b\" (UID: \"a912f526-ee89-4979-b6f7-26767d95858b\") "
Sep 10 15:49:38.672433 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.672311    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a912f526-ee89-4979-b6f7-26767d95858b-extensions-socket-volume\") pod \"a912f526-ee89-4979-b6f7-26767d95858b\" (UID: \"a912f526-ee89-4979-b6f7-26767d95858b\") "
Sep 10 15:49:38.672828 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.672794    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a912f526-ee89-4979-b6f7-26767d95858b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a912f526-ee89-4979-b6f7-26767d95858b" (UID: "a912f526-ee89-4979-b6f7-26767d95858b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:49:38.674662 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.674632    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a912f526-ee89-4979-b6f7-26767d95858b-kube-api-access-8czmp" (OuterVolumeSpecName: "kube-api-access-8czmp") pod "a912f526-ee89-4979-b6f7-26767d95858b" (UID: "a912f526-ee89-4979-b6f7-26767d95858b"). InnerVolumeSpecName "kube-api-access-8czmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:49:38.747518 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.747471    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:38.772823 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.772791    2527 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a912f526-ee89-4979-b6f7-26767d95858b-extensions-socket-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:49:38.772823 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:38.772820    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8czmp\" (UniqueName: \"kubernetes.io/projected/a912f526-ee89-4979-b6f7-26767d95858b-kube-api-access-8czmp\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:49:39.025882 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:39.025777    2527 generic.go:356] "Generic (PLEG): container finished" podID="a912f526-ee89-4979-b6f7-26767d95858b" containerID="7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98" exitCode=0
Sep 10 15:49:39.025882 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:39.025825    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-klskt"
Sep 10 15:49:39.025882 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:39.025883    2527 scope.go:117] "RemoveContainer" containerID="7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98"
Sep 10 15:49:39.034989 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:39.034955    2527 scope.go:117] "RemoveContainer" containerID="7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98"
Sep 10 15:49:39.035320 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:49:39.035303    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98\": container with ID starting with 7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98 not found: ID does not exist" containerID="7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98"
Sep 10 15:49:39.035384 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:39.035329    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98"} err="failed to get container status \"7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98\": rpc error: code = NotFound desc = could not find container \"7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98\": container with ID starting with 7dce174f84f2728ebb08608d6cef0d674c3d7dd06921220779a0800818b51a98 not found: ID does not exist"
Sep 10 15:49:39.043582 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:39.043441    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"]
Sep 10 15:49:39.046975 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:49:39.046945    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d76e8ac_f21e_46aa_a1b0_4923a5b5f982.slice/crio-cf147d632119682e2a6a7a16580937731aafb4dc53e825fe9fbc92513756281f WatchSource:0}: Error finding container cf147d632119682e2a6a7a16580937731aafb4dc53e825fe9fbc92513756281f: Status 404 returned error can't find the container with id cf147d632119682e2a6a7a16580937731aafb4dc53e825fe9fbc92513756281f
Sep 10 15:49:40.033722 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:40.033680    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr" event={"ID":"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982","Type":"ContainerStarted","Data":"2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4"}
Sep 10 15:49:40.034227 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:40.033749    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr" event={"ID":"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982","Type":"ContainerStarted","Data":"cf147d632119682e2a6a7a16580937731aafb4dc53e825fe9fbc92513756281f"}
Sep 10 15:49:40.034227 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:40.033800    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:40.564890 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:40.564840    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a912f526-ee89-4979-b6f7-26767d95858b" path="/var/lib/kubelet/pods/a912f526-ee89-4979-b6f7-26767d95858b/volumes"
Sep 10 15:49:40.989001 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:40.988922    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-t6nlf"
Sep 10 15:49:41.009095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:41.009033    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr" podStartSLOduration=3.009014772 podStartE2EDuration="3.009014772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:49:40.062226226 +0000 UTC m=+475.978351126" watchObservedRunningTime="2026-09-10 15:49:41.009014772 +0000 UTC m=+476.925139654"
Sep 10 15:49:51.044817 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:51.044784    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:56.117976 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:56.117942    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"]
Sep 10 15:49:56.118511 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:56.118143    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr" podUID="3d76e8ac-f21e-46aa-a1b0-4923a5b5f982" containerName="manager" containerID="cri-o://2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4" gracePeriod=10
Sep 10 15:49:56.376163 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:56.376084    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:56.413952 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:56.413921    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-extensions-socket-volume\") pod \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\" (UID: \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\") "
Sep 10 15:49:56.414142 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:56.413964    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs2rx\" (UniqueName: \"kubernetes.io/projected/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-kube-api-access-zs2rx\") pod \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\" (UID: \"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982\") "
Sep 10 15:49:56.416101 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:56.414626    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3d76e8ac-f21e-46aa-a1b0-4923a5b5f982" (UID: "3d76e8ac-f21e-46aa-a1b0-4923a5b5f982"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:49:56.420187 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:56.417926    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-kube-api-access-zs2rx" (OuterVolumeSpecName: "kube-api-access-zs2rx") pod "3d76e8ac-f21e-46aa-a1b0-4923a5b5f982" (UID: "3d76e8ac-f21e-46aa-a1b0-4923a5b5f982"). InnerVolumeSpecName "kube-api-access-zs2rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:49:56.517377 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:56.517315    2527 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-extensions-socket-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:49:56.517377 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:56.517368    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs2rx\" (UniqueName: \"kubernetes.io/projected/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982-kube-api-access-zs2rx\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:49:57.123954 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:57.123913    2527 generic.go:356] "Generic (PLEG): container finished" podID="3d76e8ac-f21e-46aa-a1b0-4923a5b5f982" containerID="2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4" exitCode=0
Sep 10 15:49:57.124431 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:57.123971    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"
Sep 10 15:49:57.124431 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:57.123996    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr" event={"ID":"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982","Type":"ContainerDied","Data":"2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4"}
Sep 10 15:49:57.124431 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:57.124049    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr" event={"ID":"3d76e8ac-f21e-46aa-a1b0-4923a5b5f982","Type":"ContainerDied","Data":"cf147d632119682e2a6a7a16580937731aafb4dc53e825fe9fbc92513756281f"}
Sep 10 15:49:57.124431 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:57.124073    2527 scope.go:117] "RemoveContainer" containerID="2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4"
Sep 10 15:49:57.132729 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:57.132705    2527 scope.go:117] "RemoveContainer" containerID="2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4"
Sep 10 15:49:57.133081 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:49:57.133062    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4\": container with ID starting with 2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4 not found: ID does not exist" containerID="2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4"
Sep 10 15:49:57.133153 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:57.133092    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4"} err="failed to get container status \"2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4\": rpc error: code = NotFound desc = could not find container \"2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4\": container with ID starting with 2939ca56bdfc7657d5988d0c95abab7b5c78dcf8b07f9c08f4ef08bdd456b8f4 not found: ID does not exist"
Sep 10 15:49:57.143528 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:57.143494    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"]
Sep 10 15:49:57.146992 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:57.146965    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gbbnr"]
Sep 10 15:49:58.563942 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:49:58.563910    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d76e8ac-f21e-46aa-a1b0-4923a5b5f982" path="/var/lib/kubelet/pods/3d76e8ac-f21e-46aa-a1b0-4923a5b5f982/volumes"
Sep 10 15:50:14.384043 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.384005    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hzpfj"]
Sep 10 15:50:14.384468 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.384391    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d76e8ac-f21e-46aa-a1b0-4923a5b5f982" containerName="manager"
Sep 10 15:50:14.384468 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.384403    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d76e8ac-f21e-46aa-a1b0-4923a5b5f982" containerName="manager"
Sep 10 15:50:14.384468 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.384467    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d76e8ac-f21e-46aa-a1b0-4923a5b5f982" containerName="manager"
Sep 10 15:50:14.387410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.387393    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:14.390207 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.390184    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pj7x6\""
Sep 10 15:50:14.390624 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.390608    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 15:50:14.398340 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.398313    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hzpfj"]
Sep 10 15:50:14.476012 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.475979    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkrpv\" (UniqueName: \"kubernetes.io/projected/b3a08c3c-d722-43c1-9544-f377ea7f021f-kube-api-access-hkrpv\") pod \"limitador-limitador-8588749977-hzpfj\" (UID: \"b3a08c3c-d722-43c1-9544-f377ea7f021f\") " pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:14.476216 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.476026    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b3a08c3c-d722-43c1-9544-f377ea7f021f-config-file\") pod \"limitador-limitador-8588749977-hzpfj\" (UID: \"b3a08c3c-d722-43c1-9544-f377ea7f021f\") " pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:14.483327 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.483290    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hzpfj"]
Sep 10 15:50:14.576914 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.576874    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkrpv\" (UniqueName: \"kubernetes.io/projected/b3a08c3c-d722-43c1-9544-f377ea7f021f-kube-api-access-hkrpv\") pod \"limitador-limitador-8588749977-hzpfj\" (UID: \"b3a08c3c-d722-43c1-9544-f377ea7f021f\") " pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:14.576914 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.576921    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b3a08c3c-d722-43c1-9544-f377ea7f021f-config-file\") pod \"limitador-limitador-8588749977-hzpfj\" (UID: \"b3a08c3c-d722-43c1-9544-f377ea7f021f\") " pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:14.577584 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.577563    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b3a08c3c-d722-43c1-9544-f377ea7f021f-config-file\") pod \"limitador-limitador-8588749977-hzpfj\" (UID: \"b3a08c3c-d722-43c1-9544-f377ea7f021f\") " pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:14.585295 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.585272    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkrpv\" (UniqueName: \"kubernetes.io/projected/b3a08c3c-d722-43c1-9544-f377ea7f021f-kube-api-access-hkrpv\") pod \"limitador-limitador-8588749977-hzpfj\" (UID: \"b3a08c3c-d722-43c1-9544-f377ea7f021f\") " pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:14.698406 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.698289    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:14.980107 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:14.980073    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hzpfj"]
Sep 10 15:50:14.983041 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:50:14.983010    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a08c3c_d722_43c1_9544_f377ea7f021f.slice/crio-6b8391d35ac59f14f7cf4e83e24a540b5361914560dee20de91c658adabdc886 WatchSource:0}: Error finding container 6b8391d35ac59f14f7cf4e83e24a540b5361914560dee20de91c658adabdc886: Status 404 returned error can't find the container with id 6b8391d35ac59f14f7cf4e83e24a540b5361914560dee20de91c658adabdc886
Sep 10 15:50:15.215405 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:15.215368    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj" event={"ID":"b3a08c3c-d722-43c1-9544-f377ea7f021f","Type":"ContainerStarted","Data":"6b8391d35ac59f14f7cf4e83e24a540b5361914560dee20de91c658adabdc886"}
Sep 10 15:50:16.081466 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:50:16.081422    2527 log.go:32] "ImageFsInfo from image service failed" err="rpc error: code = Unknown desc = get image fs info unable to get usage for /var/lib/containers/storage/overlay-images: get disk usage for path /var/lib/containers/storage/overlay-images: lstat /var/lib/containers/storage/overlay-images/.tmp-images.json685847132: no such file or directory"
Sep 10 15:50:16.081845 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:50:16.081482    2527 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get imageFs stats: missing image stats: <nil>"
Sep 10 15:50:16.222420 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:16.222378    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj" event={"ID":"b3a08c3c-d722-43c1-9544-f377ea7f021f","Type":"ContainerStarted","Data":"8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce"}
Sep 10 15:50:16.222616 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:16.222507    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:16.239889 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:16.239828    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj" podStartSLOduration=1.127037117 podStartE2EDuration="2.239809643s" totalImagesPullingTime="1.112772526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:50:16.23756154 +0000 UTC m=+512.153686418" watchObservedRunningTime="2026-09-10 15:50:16.239809643 +0000 UTC m=+512.155934566"
Sep 10 15:50:27.228331 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:27.228298    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:27.488951 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:27.488864    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hzpfj"]
Sep 10 15:50:27.489128 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:27.489105    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj" podUID="b3a08c3c-d722-43c1-9544-f377ea7f021f" containerName="limitador" containerID="cri-o://8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce" gracePeriod=30
Sep 10 15:50:28.046928 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.046868    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:28.206336 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.206297    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b3a08c3c-d722-43c1-9544-f377ea7f021f-config-file\") pod \"b3a08c3c-d722-43c1-9544-f377ea7f021f\" (UID: \"b3a08c3c-d722-43c1-9544-f377ea7f021f\") "
Sep 10 15:50:28.206531 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.206509    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkrpv\" (UniqueName: \"kubernetes.io/projected/b3a08c3c-d722-43c1-9544-f377ea7f021f-kube-api-access-hkrpv\") pod \"b3a08c3c-d722-43c1-9544-f377ea7f021f\" (UID: \"b3a08c3c-d722-43c1-9544-f377ea7f021f\") "
Sep 10 15:50:28.206741 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.206716    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3a08c3c-d722-43c1-9544-f377ea7f021f-config-file" (OuterVolumeSpecName: "config-file") pod "b3a08c3c-d722-43c1-9544-f377ea7f021f" (UID: "b3a08c3c-d722-43c1-9544-f377ea7f021f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:50:28.208703 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.208672    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a08c3c-d722-43c1-9544-f377ea7f021f-kube-api-access-hkrpv" (OuterVolumeSpecName: "kube-api-access-hkrpv") pod "b3a08c3c-d722-43c1-9544-f377ea7f021f" (UID: "b3a08c3c-d722-43c1-9544-f377ea7f021f"). InnerVolumeSpecName "kube-api-access-hkrpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:50:28.283747 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.283709    2527 generic.go:356] "Generic (PLEG): container finished" podID="b3a08c3c-d722-43c1-9544-f377ea7f021f" containerID="8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce" exitCode=0
Sep 10 15:50:28.284227 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.283764    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj"
Sep 10 15:50:28.284227 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.283795    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj" event={"ID":"b3a08c3c-d722-43c1-9544-f377ea7f021f","Type":"ContainerDied","Data":"8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce"}
Sep 10 15:50:28.284227 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.283856    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hzpfj" event={"ID":"b3a08c3c-d722-43c1-9544-f377ea7f021f","Type":"ContainerDied","Data":"6b8391d35ac59f14f7cf4e83e24a540b5361914560dee20de91c658adabdc886"}
Sep 10 15:50:28.284227 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.283885    2527 scope.go:117] "RemoveContainer" containerID="8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce"
Sep 10 15:50:28.293314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.293294    2527 scope.go:117] "RemoveContainer" containerID="8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce"
Sep 10 15:50:28.293620 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:50:28.293603    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce\": container with ID starting with 8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce not found: ID does not exist" containerID="8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce"
Sep 10 15:50:28.293670 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.293628    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce"} err="failed to get container status \"8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce\": rpc error: code = NotFound desc = could not find container \"8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce\": container with ID starting with 8a95c58f292377d8192ceab618d93fb63df28cdd668bd40d6b17f968943959ce not found: ID does not exist"
Sep 10 15:50:28.304754 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.304682    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hzpfj"]
Sep 10 15:50:28.307471 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.307441    2527 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b3a08c3c-d722-43c1-9544-f377ea7f021f-config-file\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:50:28.307471 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.307468    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkrpv\" (UniqueName: \"kubernetes.io/projected/b3a08c3c-d722-43c1-9544-f377ea7f021f-kube-api-access-hkrpv\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:50:28.308023 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.308000    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hzpfj"]
Sep 10 15:50:28.563518 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:28.563439    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3a08c3c-d722-43c1-9544-f377ea7f021f" path="/var/lib/kubelet/pods/b3a08c3c-d722-43c1-9544-f377ea7f021f/volumes"
Sep 10 15:50:30.891571 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.891529    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"]
Sep 10 15:50:30.892078 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.891885    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3a08c3c-d722-43c1-9544-f377ea7f021f" containerName="limitador"
Sep 10 15:50:30.892078 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.891896    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3a08c3c-d722-43c1-9544-f377ea7f021f" containerName="limitador"
Sep 10 15:50:30.892078 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.891958    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3a08c3c-d722-43c1-9544-f377ea7f021f" containerName="limitador"
Sep 10 15:50:30.896518 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.896494    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:30.898985 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.898956    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:50:30.899115 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.899067    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 15:50:30.899191 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.899172    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 15:50:30.899629 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.899611    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-77fph\""
Sep 10 15:50:30.904228 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.904115    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"]
Sep 10 15:50:30.927102 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.927049    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9ac0f436-4b90-40f0-bb12-356c8af39150-data\") pod \"postgres-69fb4594cf-jxtlx\" (UID: \"9ac0f436-4b90-40f0-bb12-356c8af39150\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:30.927310 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:30.927183    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg46m\" (UniqueName: \"kubernetes.io/projected/9ac0f436-4b90-40f0-bb12-356c8af39150-kube-api-access-cg46m\") pod \"postgres-69fb4594cf-jxtlx\" (UID: \"9ac0f436-4b90-40f0-bb12-356c8af39150\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:31.028559 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:31.028524    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg46m\" (UniqueName: \"kubernetes.io/projected/9ac0f436-4b90-40f0-bb12-356c8af39150-kube-api-access-cg46m\") pod \"postgres-69fb4594cf-jxtlx\" (UID: \"9ac0f436-4b90-40f0-bb12-356c8af39150\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:31.028726 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:31.028585    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9ac0f436-4b90-40f0-bb12-356c8af39150-data\") pod \"postgres-69fb4594cf-jxtlx\" (UID: \"9ac0f436-4b90-40f0-bb12-356c8af39150\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:31.028940 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:31.028924    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9ac0f436-4b90-40f0-bb12-356c8af39150-data\") pod \"postgres-69fb4594cf-jxtlx\" (UID: \"9ac0f436-4b90-40f0-bb12-356c8af39150\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:31.043260 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:31.043227    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg46m\" (UniqueName: \"kubernetes.io/projected/9ac0f436-4b90-40f0-bb12-356c8af39150-kube-api-access-cg46m\") pod \"postgres-69fb4594cf-jxtlx\" (UID: \"9ac0f436-4b90-40f0-bb12-356c8af39150\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:31.208668 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:31.208579    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:31.484375 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:31.484345    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"]
Sep 10 15:50:31.487105 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:50:31.487075    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac0f436_4b90_40f0_bb12_356c8af39150.slice/crio-1f6793b48693f27845e7d0a456d0eb3b0c8db6e60d2ba9d2fd32ef157a2893ab WatchSource:0}: Error finding container 1f6793b48693f27845e7d0a456d0eb3b0c8db6e60d2ba9d2fd32ef157a2893ab: Status 404 returned error can't find the container with id 1f6793b48693f27845e7d0a456d0eb3b0c8db6e60d2ba9d2fd32ef157a2893ab
Sep 10 15:50:32.305895 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:32.305855    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx" event={"ID":"9ac0f436-4b90-40f0-bb12-356c8af39150","Type":"ContainerStarted","Data":"1f6793b48693f27845e7d0a456d0eb3b0c8db6e60d2ba9d2fd32ef157a2893ab"}
Sep 10 15:50:37.344012 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:37.343972    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx" event={"ID":"9ac0f436-4b90-40f0-bb12-356c8af39150","Type":"ContainerStarted","Data":"2a43f4698ced367f1e72f0a3cab3242c314a90bcf1357caf5bb40c0cb8b99c3f"}
Sep 10 15:50:37.344533 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:37.344104    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:37.359380 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:37.359325    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx" podStartSLOduration=2.114894858 podStartE2EDuration="7.359306556s" totalImagesPullingTime="5.244411698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:50:37.358363354 +0000 UTC m=+533.274488250" watchObservedRunningTime="2026-09-10 15:50:37.359306556 +0000 UTC m=+533.275431434"
Sep 10 15:50:43.379183 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:43.379154    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxtlx"
Sep 10 15:50:50.268282 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.268237    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cb5d5dccb-4ckpj"]
Sep 10 15:50:50.293398 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.293360    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cb5d5dccb-4ckpj"]
Sep 10 15:50:50.293554 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.293482    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:50.295925 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.295896    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 15:50:50.296078 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.295931    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 15:50:50.296078 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.295931    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 15:50:50.296565 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.296547    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bxbnj\""
Sep 10 15:50:50.413487 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.413447    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-webhook-cert\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:50.413673 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.413510    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z4hx\" (UniqueName: \"kubernetes.io/projected/d8a54997-a633-4b72-9b02-541eb8851be6-kube-api-access-4z4hx\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:50.413673 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.413625    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-metrics-certs\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:50.431078 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.431032    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69fdcbc57-vp468"]
Sep 10 15:50:50.434579 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.434552    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:50.446026 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.445996    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-vp468"]
Sep 10 15:50:50.514508 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.514463    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-metrics-certs\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:50.514508 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.514512    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-webhook-cert\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:50.514740 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.514550    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-webhook-cert\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:50.514740 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.514571    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-metrics-certs\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:50.514740 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.514606    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z4hx\" (UniqueName: \"kubernetes.io/projected/d8a54997-a633-4b72-9b02-541eb8851be6-kube-api-access-4z4hx\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:50.514740 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.514630    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf7zs\" (UniqueName: \"kubernetes.io/projected/d79009a5-1cda-4084-98c0-c9c90fdbc564-kube-api-access-kf7zs\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:50.514740 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:50:50.514646    2527 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 15:50:50.514740 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:50:50.514734    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-metrics-certs podName:d8a54997-a633-4b72-9b02-541eb8851be6 nodeName:}" failed. No retries permitted until 2026-09-10 15:50:51.014713175 +0000 UTC m=+546.930838039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-metrics-certs") pod "maas-controller-7cb5d5dccb-4ckpj" (UID: "d8a54997-a633-4b72-9b02-541eb8851be6") : secret "maas-controller-metrics-tls" not found
Sep 10 15:50:50.517164 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.517124    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-webhook-cert\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:50.524257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.524182    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z4hx\" (UniqueName: \"kubernetes.io/projected/d8a54997-a633-4b72-9b02-541eb8851be6-kube-api-access-4z4hx\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:50.615607 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.615564    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-webhook-cert\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:50.615815 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.615619    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-metrics-certs\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:50.615815 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.615672    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf7zs\" (UniqueName: \"kubernetes.io/projected/d79009a5-1cda-4084-98c0-c9c90fdbc564-kube-api-access-kf7zs\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:50.615939 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:50:50.615810    2527 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 15:50:50.615939 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:50:50.615893    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-metrics-certs podName:d79009a5-1cda-4084-98c0-c9c90fdbc564 nodeName:}" failed. No retries permitted until 2026-09-10 15:50:51.115870279 +0000 UTC m=+547.031995153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-metrics-certs") pod "maas-controller-69fdcbc57-vp468" (UID: "d79009a5-1cda-4084-98c0-c9c90fdbc564") : secret "maas-controller-metrics-tls" not found
Sep 10 15:50:50.618344 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.618317    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-webhook-cert\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:50.624183 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:50.624152    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf7zs\" (UniqueName: \"kubernetes.io/projected/d79009a5-1cda-4084-98c0-c9c90fdbc564-kube-api-access-kf7zs\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:51.019767 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:51.019711    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-metrics-certs\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:51.022363 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:51.022335    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-metrics-certs\") pod \"maas-controller-7cb5d5dccb-4ckpj\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") " pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:51.120908 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:51.120817    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-metrics-certs\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:51.123386 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:51.123362    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-metrics-certs\") pod \"maas-controller-69fdcbc57-vp468\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") " pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:51.205453 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:51.205421    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:51.346831 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:51.346781    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:51.487551 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:51.487515    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cb5d5dccb-4ckpj"]
Sep 10 15:50:51.491375 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:50:51.491341    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a54997_a633_4b72_9b02_541eb8851be6.slice/crio-d1584cbb3dfd758fcc9e9848fae06e83abade88e802e1f6032ef10e9e1d4f11a WatchSource:0}: Error finding container d1584cbb3dfd758fcc9e9848fae06e83abade88e802e1f6032ef10e9e1d4f11a: Status 404 returned error can't find the container with id d1584cbb3dfd758fcc9e9848fae06e83abade88e802e1f6032ef10e9e1d4f11a
Sep 10 15:50:51.627763 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:51.627683    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-vp468"]
Sep 10 15:50:51.630814 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:50:51.630787    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79009a5_1cda_4084_98c0_c9c90fdbc564.slice/crio-a72b8b4c008eb2911a4e8d5c06746a03a7d3100f912d72e21bcba930bcf81ce8 WatchSource:0}: Error finding container a72b8b4c008eb2911a4e8d5c06746a03a7d3100f912d72e21bcba930bcf81ce8: Status 404 returned error can't find the container with id a72b8b4c008eb2911a4e8d5c06746a03a7d3100f912d72e21bcba930bcf81ce8
Sep 10 15:50:52.416864 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:52.416820    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj" event={"ID":"d8a54997-a633-4b72-9b02-541eb8851be6","Type":"ContainerStarted","Data":"d1584cbb3dfd758fcc9e9848fae06e83abade88e802e1f6032ef10e9e1d4f11a"}
Sep 10 15:50:52.418054 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:52.418031    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-vp468" event={"ID":"d79009a5-1cda-4084-98c0-c9c90fdbc564","Type":"ContainerStarted","Data":"a72b8b4c008eb2911a4e8d5c06746a03a7d3100f912d72e21bcba930bcf81ce8"}
Sep 10 15:50:55.436277 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:55.436239    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj" event={"ID":"d8a54997-a633-4b72-9b02-541eb8851be6","Type":"ContainerStarted","Data":"fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6"}
Sep 10 15:50:55.436735 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:55.436397    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:50:55.437903 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:55.437872    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-vp468" event={"ID":"d79009a5-1cda-4084-98c0-c9c90fdbc564","Type":"ContainerStarted","Data":"aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d"}
Sep 10 15:50:55.438004 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:55.437939    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:50:55.459355 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:55.459294    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj" podStartSLOduration=2.039016385 podStartE2EDuration="5.4592743s" totalImagesPullingTime="3.420257915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:50:55.456902158 +0000 UTC m=+551.373027060" watchObservedRunningTime="2026-09-10 15:50:55.4592743 +0000 UTC m=+551.375399179"
Sep 10 15:50:55.480063 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:50:55.480002    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69fdcbc57-vp468" podStartSLOduration=2.188221074 podStartE2EDuration="5.479980986s" totalImagesPullingTime="3.291759912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:50:55.47949808 +0000 UTC m=+551.395623007" watchObservedRunningTime="2026-09-10 15:50:55.479980986 +0000 UTC m=+551.396105866"
Sep 10 15:51:06.446217 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.446186    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:51:06.446708 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.446258    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 15:51:06.499629 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.499589    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cb5d5dccb-4ckpj"]
Sep 10 15:51:06.499853 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.499802    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj" podUID="d8a54997-a633-4b72-9b02-541eb8851be6" containerName="manager" containerID="cri-o://fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6" gracePeriod=30
Sep 10 15:51:06.759366 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.759339    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:51:06.763532 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.763509    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z4hx\" (UniqueName: \"kubernetes.io/projected/d8a54997-a633-4b72-9b02-541eb8851be6-kube-api-access-4z4hx\") pod \"d8a54997-a633-4b72-9b02-541eb8851be6\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") "
Sep 10 15:51:06.763636 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.763584    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-metrics-certs\") pod \"d8a54997-a633-4b72-9b02-541eb8851be6\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") "
Sep 10 15:51:06.763636 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.763606    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-webhook-cert\") pod \"d8a54997-a633-4b72-9b02-541eb8851be6\" (UID: \"d8a54997-a633-4b72-9b02-541eb8851be6\") "
Sep 10 15:51:06.765865 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.765829    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d8a54997-a633-4b72-9b02-541eb8851be6" (UID: "d8a54997-a633-4b72-9b02-541eb8851be6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:51:06.765979 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.765883    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8a54997-a633-4b72-9b02-541eb8851be6-kube-api-access-4z4hx" (OuterVolumeSpecName: "kube-api-access-4z4hx") pod "d8a54997-a633-4b72-9b02-541eb8851be6" (UID: "d8a54997-a633-4b72-9b02-541eb8851be6"). InnerVolumeSpecName "kube-api-access-4z4hx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:51:06.765979 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.765928    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d8a54997-a633-4b72-9b02-541eb8851be6" (UID: "d8a54997-a633-4b72-9b02-541eb8851be6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:51:06.864310 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.864270    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z4hx\" (UniqueName: \"kubernetes.io/projected/d8a54997-a633-4b72-9b02-541eb8851be6-kube-api-access-4z4hx\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:51:06.864310 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.864302    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:51:06.864310 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:06.864313    2527 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8a54997-a633-4b72-9b02-541eb8851be6-webhook-cert\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:51:07.497288 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:07.497253    2527 generic.go:356] "Generic (PLEG): container finished" podID="d8a54997-a633-4b72-9b02-541eb8851be6" containerID="fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6" exitCode=0
Sep 10 15:51:07.497288 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:07.497291    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj" event={"ID":"d8a54997-a633-4b72-9b02-541eb8851be6","Type":"ContainerDied","Data":"fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6"}
Sep 10 15:51:07.497775 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:07.497300    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj"
Sep 10 15:51:07.497775 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:07.497335    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cb5d5dccb-4ckpj" event={"ID":"d8a54997-a633-4b72-9b02-541eb8851be6","Type":"ContainerDied","Data":"d1584cbb3dfd758fcc9e9848fae06e83abade88e802e1f6032ef10e9e1d4f11a"}
Sep 10 15:51:07.497775 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:07.497359    2527 scope.go:117] "RemoveContainer" containerID="fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6"
Sep 10 15:51:07.507192 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:07.507023    2527 scope.go:117] "RemoveContainer" containerID="fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6"
Sep 10 15:51:07.507456 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:51:07.507430    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6\": container with ID starting with fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6 not found: ID does not exist" containerID="fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6"
Sep 10 15:51:07.507519 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:07.507469    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6"} err="failed to get container status \"fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6\": rpc error: code = NotFound desc = could not find container \"fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6\": container with ID starting with fbc15e481cf74113c74c381c59ddabe2a045848b69fb330294aa511f5ea40ac6 not found: ID does not exist"
Sep 10 15:51:07.521421 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:07.521385    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cb5d5dccb-4ckpj"]
Sep 10 15:51:07.526556 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:07.526528    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cb5d5dccb-4ckpj"]
Sep 10 15:51:08.563480 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:08.563434    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8a54997-a633-4b72-9b02-541eb8851be6" path="/var/lib/kubelet/pods/d8a54997-a633-4b72-9b02-541eb8851be6/volumes"
Sep 10 15:51:27.707181 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.707145    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6689875f55-k25tg"]
Sep 10 15:51:27.707691 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.707675    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a54997-a633-4b72-9b02-541eb8851be6" containerName="manager"
Sep 10 15:51:27.707745 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.707693    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a54997-a633-4b72-9b02-541eb8851be6" containerName="manager"
Sep 10 15:51:27.707788 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.707778    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8a54997-a633-4b72-9b02-541eb8851be6" containerName="manager"
Sep 10 15:51:27.713911 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.713885    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.716523 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.716493    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 15:51:27.716523 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.716504    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 15:51:27.716748 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.716597    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nwclg\""
Sep 10 15:51:27.722426 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.722268    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6689875f55-k25tg"]
Sep 10 15:51:27.837899 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.837858    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-maas-api-tls\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.838119 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.837968    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27cgw\" (UniqueName: \"kubernetes.io/projected/bb39753e-59c7-4597-82be-dad7fb95eba3-kube-api-access-27cgw\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.838119 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.838012    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb39753e-59c7-4597-82be-dad7fb95eba3-openshift-service-ca\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.838119 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.838077    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-metrics-certs\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.938877 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.938823    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-maas-api-tls\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.939073 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.938900    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27cgw\" (UniqueName: \"kubernetes.io/projected/bb39753e-59c7-4597-82be-dad7fb95eba3-kube-api-access-27cgw\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.939073 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.938932    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb39753e-59c7-4597-82be-dad7fb95eba3-openshift-service-ca\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.939073 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:51:27.938992    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 10 15:51:27.939073 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:51:27.939063    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-maas-api-tls podName:bb39753e-59c7-4597-82be-dad7fb95eba3 nodeName:}" failed. No retries permitted until 2026-09-10 15:51:28.43904068 +0000 UTC m=+584.355165554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-maas-api-tls") pod "maas-api-6689875f55-k25tg" (UID: "bb39753e-59c7-4597-82be-dad7fb95eba3") : secret "maas-api-serving-cert" not found
Sep 10 15:51:27.939309 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.939092    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-metrics-certs\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.939309 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:51:27.939215    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 15:51:27.939309 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:51:27.939293    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-metrics-certs podName:bb39753e-59c7-4597-82be-dad7fb95eba3 nodeName:}" failed. No retries permitted until 2026-09-10 15:51:28.439271947 +0000 UTC m=+584.355396810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-metrics-certs") pod "maas-api-6689875f55-k25tg" (UID: "bb39753e-59c7-4597-82be-dad7fb95eba3") : secret "maas-api-metrics-tls" not found
Sep 10 15:51:27.939608 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.939590    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb39753e-59c7-4597-82be-dad7fb95eba3-openshift-service-ca\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:27.949902 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:27.949862    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27cgw\" (UniqueName: \"kubernetes.io/projected/bb39753e-59c7-4597-82be-dad7fb95eba3-kube-api-access-27cgw\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:28.354918 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.354882    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-88prc"]
Sep 10 15:51:28.359094 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.359063    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:28.362597 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.362563    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 15:51:28.362755 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.362676    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2x5ws\""
Sep 10 15:51:28.370830 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.370800    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-88prc"]
Sep 10 15:51:28.443013 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.442977    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-metrics-certs\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:28.443223 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.443048    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8cb2ee3-5670-4eb0-b31d-afa362698dd5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-88prc\" (UID: \"e8cb2ee3-5670-4eb0-b31d-afa362698dd5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:28.443223 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.443080    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-maas-api-tls\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:28.443223 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.443125    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wmfr\" (UniqueName: \"kubernetes.io/projected/e8cb2ee3-5670-4eb0-b31d-afa362698dd5-kube-api-access-5wmfr\") pod \"payload-processing-5b54b7fbb7-88prc\" (UID: \"e8cb2ee3-5670-4eb0-b31d-afa362698dd5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:28.445635 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.445610    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-metrics-certs\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:28.445635 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.445628    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-maas-api-tls\") pod \"maas-api-6689875f55-k25tg\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") " pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:28.544346 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.544303    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wmfr\" (UniqueName: \"kubernetes.io/projected/e8cb2ee3-5670-4eb0-b31d-afa362698dd5-kube-api-access-5wmfr\") pod \"payload-processing-5b54b7fbb7-88prc\" (UID: \"e8cb2ee3-5670-4eb0-b31d-afa362698dd5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:28.544538 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.544420    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8cb2ee3-5670-4eb0-b31d-afa362698dd5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-88prc\" (UID: \"e8cb2ee3-5670-4eb0-b31d-afa362698dd5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:28.545205 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.545183    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8cb2ee3-5670-4eb0-b31d-afa362698dd5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-88prc\" (UID: \"e8cb2ee3-5670-4eb0-b31d-afa362698dd5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:28.557426 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.557392    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wmfr\" (UniqueName: \"kubernetes.io/projected/e8cb2ee3-5670-4eb0-b31d-afa362698dd5-kube-api-access-5wmfr\") pod \"payload-processing-5b54b7fbb7-88prc\" (UID: \"e8cb2ee3-5670-4eb0-b31d-afa362698dd5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:28.627447 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.627355    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:28.675169 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.675115    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:28.909591 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.909562    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6689875f55-k25tg"]
Sep 10 15:51:28.912283 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:51:28.912254    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb39753e_59c7_4597_82be_dad7fb95eba3.slice/crio-d45d2baf520948e6889ef813b4c27f3f98745fddeec3a7a273c5d64f2f89d423 WatchSource:0}: Error finding container d45d2baf520948e6889ef813b4c27f3f98745fddeec3a7a273c5d64f2f89d423: Status 404 returned error can't find the container with id d45d2baf520948e6889ef813b4c27f3f98745fddeec3a7a273c5d64f2f89d423
Sep 10 15:51:28.959849 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:28.959800    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-88prc"]
Sep 10 15:51:28.963741 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:51:28.963698    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8cb2ee3_5670_4eb0_b31d_afa362698dd5.slice/crio-d67f88d3124c2056074dc106059ba1d53e3aad54b0b9a1c1095855a55ad89682 WatchSource:0}: Error finding container d67f88d3124c2056074dc106059ba1d53e3aad54b0b9a1c1095855a55ad89682: Status 404 returned error can't find the container with id d67f88d3124c2056074dc106059ba1d53e3aad54b0b9a1c1095855a55ad89682
Sep 10 15:51:29.616424 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:29.616347    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg" event={"ID":"bb39753e-59c7-4597-82be-dad7fb95eba3","Type":"ContainerStarted","Data":"d45d2baf520948e6889ef813b4c27f3f98745fddeec3a7a273c5d64f2f89d423"}
Sep 10 15:51:29.619336 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:29.619303    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc" event={"ID":"e8cb2ee3-5670-4eb0-b31d-afa362698dd5","Type":"ContainerStarted","Data":"d67f88d3124c2056074dc106059ba1d53e3aad54b0b9a1c1095855a55ad89682"}
Sep 10 15:51:31.642473 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:31.642426    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg" event={"ID":"bb39753e-59c7-4597-82be-dad7fb95eba3","Type":"ContainerStarted","Data":"fb919b4f3517bcfa8423b2e75c0d0595bb995029537b5bf73e26426841143c1d"}
Sep 10 15:51:31.642947 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:31.642593    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:31.644824 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:31.644781    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc" event={"ID":"e8cb2ee3-5670-4eb0-b31d-afa362698dd5","Type":"ContainerStarted","Data":"2e48666b9f28abaa95ee8e220f16fdd64d1f5a18cc08831bb9beaf0449afa866"}
Sep 10 15:51:31.644995 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:31.644937    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:31.660717 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:31.660662    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg" podStartSLOduration=2.034555603 podStartE2EDuration="4.660644945s" totalImagesPullingTime="2.626089342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:51:31.658892289 +0000 UTC m=+587.575017187" watchObservedRunningTime="2026-09-10 15:51:31.660644945 +0000 UTC m=+587.576769824"
Sep 10 15:51:31.677539 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:31.677480    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc" podStartSLOduration=1.0805311930000001 podStartE2EDuration="3.677463334s" totalImagesPullingTime="2.596932141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:51:31.675343934 +0000 UTC m=+587.591468836" watchObservedRunningTime="2026-09-10 15:51:31.677463334 +0000 UTC m=+587.593588213"
Sep 10 15:51:37.656204 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:37.656175    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 15:51:42.652085 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:42.652050    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-88prc"
Sep 10 15:51:44.519264 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:44.519237    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 15:51:44.519708 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:44.519304    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 15:51:49.666270 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.666238    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"]
Sep 10 15:51:49.669049 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.669021    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.672106 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.672076    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 15:51:49.672265 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.672129    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4b9fq\""
Sep 10 15:51:49.672899 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.672873    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 15:51:49.673033 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.672980    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 15:51:49.681280 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.681251    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"]
Sep 10 15:51:49.842937 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.842898    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdh82\" (UniqueName: \"kubernetes.io/projected/48825b55-da89-4817-bc4d-dcecf274ab67-kube-api-access-qdh82\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.843118 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.842949    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.843118 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.843009    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.843118 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.843046    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.843118 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.843078    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.843118 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.843099    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48825b55-da89-4817-bc4d-dcecf274ab67-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.943700 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.943598    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.943700 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.943638    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48825b55-da89-4817-bc4d-dcecf274ab67-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.943919 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.943738    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdh82\" (UniqueName: \"kubernetes.io/projected/48825b55-da89-4817-bc4d-dcecf274ab67-kube-api-access-qdh82\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.943919 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.943870    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.943919 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.943909    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.944074 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.943926    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.944074 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.944050    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.944262 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.944239    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.944360 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.944342    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.946233 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.946208    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48825b55-da89-4817-bc4d-dcecf274ab67-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.946408 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.946390    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48825b55-da89-4817-bc4d-dcecf274ab67-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.951614 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.951592    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdh82\" (UniqueName: \"kubernetes.io/projected/48825b55-da89-4817-bc4d-dcecf274ab67-kube-api-access-qdh82\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5\" (UID: \"48825b55-da89-4817-bc4d-dcecf274ab67\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:49.980747 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:49.980707    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:50.256439 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:50.256413    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"]
Sep 10 15:51:50.258861 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:51:50.258833    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48825b55_da89_4817_bc4d_dcecf274ab67.slice/crio-7ae5cf047dab906f9bf0c32b8832a14043eb70f8cc06c88ba5c0c31dfabc8166 WatchSource:0}: Error finding container 7ae5cf047dab906f9bf0c32b8832a14043eb70f8cc06c88ba5c0c31dfabc8166: Status 404 returned error can't find the container with id 7ae5cf047dab906f9bf0c32b8832a14043eb70f8cc06c88ba5c0c31dfabc8166
Sep 10 15:51:50.745303 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:50.745256    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5" event={"ID":"48825b55-da89-4817-bc4d-dcecf274ab67","Type":"ContainerStarted","Data":"7ae5cf047dab906f9bf0c32b8832a14043eb70f8cc06c88ba5c0c31dfabc8166"}
Sep 10 15:51:51.751411 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:51.751315    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5" event={"ID":"48825b55-da89-4817-bc4d-dcecf274ab67","Type":"ContainerStarted","Data":"1e30e2fc8b2de6ad023a76c2a3c4119960009f214f080849679b69d8953ae6b0"}
Sep 10 15:51:51.768062 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:51.768004    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5" podStartSLOduration=1.749621758 podStartE2EDuration="2.767984552s" totalImagesPullingTime="1.018362794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:51:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:51:51.76728389 +0000 UTC m=+607.683408790" watchObservedRunningTime="2026-09-10 15:51:51.767984552 +0000 UTC m=+607.684109432"
Sep 10 15:51:59.981836 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:59.981792    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:59.982281 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:59.981868    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:51:59.994888 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:51:59.994851    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:52:00.806959 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:00.806928    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5"
Sep 10 15:52:16.250569 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.250527    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"]
Sep 10 15:52:16.254318 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.254294    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.256484 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.256455    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 15:52:16.265441 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.265409    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"]
Sep 10 15:52:16.382605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.382569    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.382605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.382618    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.382836 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.382650    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76rjl\" (UniqueName: \"kubernetes.io/projected/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-kube-api-access-76rjl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.382836 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.382743    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.382910 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.382858    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.382910 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.382898    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.483941 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.483893    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.484257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.483951    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.484257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.483995    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.484257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.484026    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76rjl\" (UniqueName: \"kubernetes.io/projected/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-kube-api-access-76rjl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.484257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.484059    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.484257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.484150    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.484533 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.484509    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.484596 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.484512    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.484648 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.484603    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.486370 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.486351    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.486698 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.486681    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.492561 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.492529    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76rjl\" (UniqueName: \"kubernetes.io/projected/8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d-kube-api-access-76rjl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj\" (UID: \"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.567981 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.567950    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:16.844422 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.844391    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"]
Sep 10 15:52:16.847319 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:52:16.847285    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d352c83_b914_4ad8_b0ff_5d4bb8c6a39d.slice/crio-e7d9028122251c0e7fb1baec2b8d560875d206bce01eef101f8aaba9bcedbaf5 WatchSource:0}: Error finding container e7d9028122251c0e7fb1baec2b8d560875d206bce01eef101f8aaba9bcedbaf5: Status 404 returned error can't find the container with id e7d9028122251c0e7fb1baec2b8d560875d206bce01eef101f8aaba9bcedbaf5
Sep 10 15:52:16.876899 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:16.876861    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj" event={"ID":"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d","Type":"ContainerStarted","Data":"e7d9028122251c0e7fb1baec2b8d560875d206bce01eef101f8aaba9bcedbaf5"}
Sep 10 15:52:17.884184 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:17.884127    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj" event={"ID":"8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d","Type":"ContainerStarted","Data":"783fc23ee2f6a54e127adf403e68248d8091452c343e1e013bb2d14df57a6751"}
Sep 10 15:52:17.900867 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:17.900801    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj" podStartSLOduration=1.7093556140000001 podStartE2EDuration="1.900747648s" totalImagesPullingTime="191.392034ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:52:17.899745029 +0000 UTC m=+633.815869908" watchObservedRunningTime="2026-09-10 15:52:17.900747648 +0000 UTC m=+633.816872527"
Sep 10 15:52:20.948990 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:20.948950    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"]
Sep 10 15:52:20.952790 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:20.952762    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:20.954885 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:20.954857    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 15:52:20.961579 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:20.961555    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"]
Sep 10 15:52:21.025064 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.025022    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.025605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.025089    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b234007a-b9d3-491a-8e8e-2080339aa97f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.025605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.025198    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.025605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.025243    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.025605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.025372    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.025605 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.025472    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwlzs\" (UniqueName: \"kubernetes.io/projected/b234007a-b9d3-491a-8e8e-2080339aa97f-kube-api-access-mwlzs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.126404 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.126363    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwlzs\" (UniqueName: \"kubernetes.io/projected/b234007a-b9d3-491a-8e8e-2080339aa97f-kube-api-access-mwlzs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.126633 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.126428    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.126633 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.126469    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b234007a-b9d3-491a-8e8e-2080339aa97f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.126633 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.126505    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.126633 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.126528    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.126633 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.126579    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.126944 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.126917    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.127006 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.126948    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.127246 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.127224    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.129302 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.129277    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b234007a-b9d3-491a-8e8e-2080339aa97f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.129401 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.129377    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b234007a-b9d3-491a-8e8e-2080339aa97f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.135383 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.135360    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwlzs\" (UniqueName: \"kubernetes.io/projected/b234007a-b9d3-491a-8e8e-2080339aa97f-kube-api-access-mwlzs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q9lhv\" (UID: \"b234007a-b9d3-491a-8e8e-2080339aa97f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.265647 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.265550    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:21.543448 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.543421    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"]
Sep 10 15:52:21.545607 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:52:21.545569    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb234007a_b9d3_491a_8e8e_2080339aa97f.slice/crio-9992ab24cc17ac75dd91beb25e60bad40362ecc72f811b528a056d81bfaa93bc WatchSource:0}: Error finding container 9992ab24cc17ac75dd91beb25e60bad40362ecc72f811b528a056d81bfaa93bc: Status 404 returned error can't find the container with id 9992ab24cc17ac75dd91beb25e60bad40362ecc72f811b528a056d81bfaa93bc
Sep 10 15:52:21.906215 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.906175    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv" event={"ID":"b234007a-b9d3-491a-8e8e-2080339aa97f","Type":"ContainerStarted","Data":"74805e4d754b11ee55604cba3edf7cf245c54c725af6a5f67c879b408b720a46"}
Sep 10 15:52:21.906417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.906237    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv" event={"ID":"b234007a-b9d3-491a-8e8e-2080339aa97f","Type":"ContainerStarted","Data":"9992ab24cc17ac75dd91beb25e60bad40362ecc72f811b528a056d81bfaa93bc"}
Sep 10 15:52:21.923062 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:21.922992    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv" podStartSLOduration=1.6973790659999999 podStartE2EDuration="1.922974421s" totalImagesPullingTime="225.595355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:52:21.922496456 +0000 UTC m=+637.838621352" watchObservedRunningTime="2026-09-10 15:52:21.922974421 +0000 UTC m=+637.839099300"
Sep 10 15:52:26.568731 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:26.568691    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:26.569182 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:26.568767    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:26.581601 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:26.581564    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:26.944371 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:26.944292    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj"
Sep 10 15:52:31.266515 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:31.266482    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:31.266980 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:31.266547    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:31.279559 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:31.279531    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:52:31.970410 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:52:31.970374    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q9lhv"
Sep 10 15:55:08.076172 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.076065    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"]
Sep 10 15:55:08.080384 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.080347    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.084165 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.084122    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-789be7-serving-cert\""
Sep 10 15:55:08.094640 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.094612    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"]
Sep 10 15:55:08.242127 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.242088    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-maas-api-tls\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.242339 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.242174    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-metrics-certs\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.242339 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.242195    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/453b04b9-0a28-4db4-8fa9-77d296992e3d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.242339 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.242235    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5wgh\" (UniqueName: \"kubernetes.io/projected/453b04b9-0a28-4db4-8fa9-77d296992e3d-kube-api-access-p5wgh\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.343369 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.343273    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-maas-api-tls\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.343369 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.343366    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-metrics-certs\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.343624 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.343399    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/453b04b9-0a28-4db4-8fa9-77d296992e3d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.343624 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:08.343451    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-serving-cert: secret "maas-api-e2e-inf-a-789be7-serving-cert" not found
Sep 10 15:55:08.343624 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.343464    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5wgh\" (UniqueName: \"kubernetes.io/projected/453b04b9-0a28-4db4-8fa9-77d296992e3d-kube-api-access-p5wgh\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.343624 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:08.343538    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-maas-api-tls podName:453b04b9-0a28-4db4-8fa9-77d296992e3d nodeName:}" failed. No retries permitted until 2026-09-10 15:55:08.843515434 +0000 UTC m=+804.759640346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-maas-api-tls") pod "maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb" (UID: "453b04b9-0a28-4db4-8fa9-77d296992e3d") : secret "maas-api-e2e-inf-a-789be7-serving-cert" not found
Sep 10 15:55:08.344422 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.344394    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/453b04b9-0a28-4db4-8fa9-77d296992e3d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.346309 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.346275    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-metrics-certs\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.352744 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.352711    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5wgh\" (UniqueName: \"kubernetes.io/projected/453b04b9-0a28-4db4-8fa9-77d296992e3d-kube-api-access-p5wgh\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.849255 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.849215    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-maas-api-tls\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.851885 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.851849    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-maas-api-tls\") pod \"maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:08.992276 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:08.992228    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:09.275265 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:09.275229    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"]
Sep 10 15:55:09.277415 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:55:09.277372    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod453b04b9_0a28_4db4_8fa9_77d296992e3d.slice/crio-c025987af31bc671565fd1437fe0daffecc319759439c7b6a11ae5cbbf69234e WatchSource:0}: Error finding container c025987af31bc671565fd1437fe0daffecc319759439c7b6a11ae5cbbf69234e: Status 404 returned error can't find the container with id c025987af31bc671565fd1437fe0daffecc319759439c7b6a11ae5cbbf69234e
Sep 10 15:55:09.278883 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:09.278866    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:55:09.769891 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:09.769851    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb" event={"ID":"453b04b9-0a28-4db4-8fa9-77d296992e3d","Type":"ContainerStarted","Data":"c025987af31bc671565fd1437fe0daffecc319759439c7b6a11ae5cbbf69234e"}
Sep 10 15:55:10.776807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:10.776761    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb" event={"ID":"453b04b9-0a28-4db4-8fa9-77d296992e3d","Type":"ContainerStarted","Data":"d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66"}
Sep 10 15:55:10.777233 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:10.776833    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:10.796590 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:10.796526    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb" podStartSLOduration=2.371204599 podStartE2EDuration="2.796504198s" totalImagesPullingTime="425.299599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:55:10.794467417 +0000 UTC m=+806.710592295" watchObservedRunningTime="2026-09-10 15:55:10.796504198 +0000 UTC m=+806.712629079"
Sep 10 15:55:15.500052 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.500008    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"]
Sep 10 15:55:15.504626 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.504596    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.507494 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.507456    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-941c6070-serving-cert\""
Sep 10 15:55:15.515660 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.515215    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"]
Sep 10 15:55:15.618609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.618563    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-maas-api-tls\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.618796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.618640    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-metrics-certs\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.618796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.618689    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m799d\" (UniqueName: \"kubernetes.io/projected/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-kube-api-access-m799d\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.618796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.618717    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-openshift-service-ca\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.720303 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.720250    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-metrics-certs\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.720510 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.720345    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m799d\" (UniqueName: \"kubernetes.io/projected/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-kube-api-access-m799d\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.720510 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.720371    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-openshift-service-ca\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.720510 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.720433    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-maas-api-tls\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.720640 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:15.720547    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-serving-cert: secret "maas-api-e2e-ait-941c6070-serving-cert" not found
Sep 10 15:55:15.721250 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:15.721081    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-maas-api-tls podName:3d94a7a2-71fb-4aff-a945-f0d9e58824c2 nodeName:}" failed. No retries permitted until 2026-09-10 15:55:16.221023816 +0000 UTC m=+812.137148682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-maas-api-tls") pod "maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx" (UID: "3d94a7a2-71fb-4aff-a945-f0d9e58824c2") : secret "maas-api-e2e-ait-941c6070-serving-cert" not found
Sep 10 15:55:15.721976 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.721937    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-openshift-service-ca\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.730223 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.730187    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-metrics-certs\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:15.733576 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:15.733542    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m799d\" (UniqueName: \"kubernetes.io/projected/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-kube-api-access-m799d\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:16.224284 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.224235    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-maas-api-tls\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:16.226810 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.226787    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-maas-api-tls\") pod \"maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:16.369190 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.369153    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"]
Sep 10 15:55:16.372923 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.372905    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:16.375716 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.375680    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-941c6070-dockercfg-6gxx6\""
Sep 10 15:55:16.375869 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.375681    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-941c6070\""
Sep 10 15:55:16.384583 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.384556    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"]
Sep 10 15:55:16.420927 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.420892    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:16.526807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.526759    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwf98\" (UniqueName: \"kubernetes.io/projected/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-kube-api-access-cwf98\") pod \"payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h\" (UID: \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:16.527263 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.526828    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h\" (UID: \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:16.627783 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.627739    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwf98\" (UniqueName: \"kubernetes.io/projected/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-kube-api-access-cwf98\") pod \"payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h\" (UID: \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:16.628034 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.627801    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h\" (UID: \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:16.628956 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.628931    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h\" (UID: \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:16.635876 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.635844    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwf98\" (UniqueName: \"kubernetes.io/projected/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-kube-api-access-cwf98\") pod \"payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h\" (UID: \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:16.684823 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.684784    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:16.711305 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.711280    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"]
Sep 10 15:55:16.714733 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:55:16.714699    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d94a7a2_71fb_4aff_a945_f0d9e58824c2.slice/crio-dc2dc34029493fb0ac77fd27203622609c8ece84845abab1fb75a30f82215965 WatchSource:0}: Error finding container dc2dc34029493fb0ac77fd27203622609c8ece84845abab1fb75a30f82215965: Status 404 returned error can't find the container with id dc2dc34029493fb0ac77fd27203622609c8ece84845abab1fb75a30f82215965
Sep 10 15:55:16.789117 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.789088    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:55:16.811059 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.811018    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx" event={"ID":"3d94a7a2-71fb-4aff-a945-f0d9e58824c2","Type":"ContainerStarted","Data":"dc2dc34029493fb0ac77fd27203622609c8ece84845abab1fb75a30f82215965"}
Sep 10 15:55:16.968478 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:16.968449    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"]
Sep 10 15:55:16.970806 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:55:16.970771    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod368ab3ac_d0cc_49bf_b82d_d862fc5f8c0e.slice/crio-483291aeea2aa08bf46631042543da8aa99353933e47d1f7f72e25607308f5e8 WatchSource:0}: Error finding container 483291aeea2aa08bf46631042543da8aa99353933e47d1f7f72e25607308f5e8: Status 404 returned error can't find the container with id 483291aeea2aa08bf46631042543da8aa99353933e47d1f7f72e25607308f5e8
Sep 10 15:55:17.819628 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:17.819586    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h" event={"ID":"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e","Type":"ContainerStarted","Data":"3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521"}
Sep 10 15:55:17.820086 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:17.819660    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h" event={"ID":"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e","Type":"ContainerStarted","Data":"483291aeea2aa08bf46631042543da8aa99353933e47d1f7f72e25607308f5e8"}
Sep 10 15:55:17.820086 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:17.819784    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:17.821730 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:17.821704    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx" event={"ID":"3d94a7a2-71fb-4aff-a945-f0d9e58824c2","Type":"ContainerStarted","Data":"fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e"}
Sep 10 15:55:17.821855 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:17.821840    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:17.840866 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:17.840799    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h" podStartSLOduration=1.840783767 podStartE2EDuration="1.840783767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:55:17.838418414 +0000 UTC m=+813.754543297" watchObservedRunningTime="2026-09-10 15:55:17.840783767 +0000 UTC m=+813.756908712"
Sep 10 15:55:17.857311 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:17.857247    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx" podStartSLOduration=2.534765653 podStartE2EDuration="2.857228337s" totalImagesPullingTime="322.462684ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:55:17.854478437 +0000 UTC m=+813.770603331" watchObservedRunningTime="2026-09-10 15:55:17.857228337 +0000 UTC m=+813.773353217"
Sep 10 15:55:19.487174 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.487118    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"]
Sep 10 15:55:19.491801 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.491772    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.494210 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.494178    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-314824-serving-cert\""
Sep 10 15:55:19.504247 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.504216    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"]
Sep 10 15:55:19.659095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.659044    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b92nf\" (UniqueName: \"kubernetes.io/projected/44e4d998-cc95-4e59-9192-07bc8417c31c-kube-api-access-b92nf\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.659095 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.659093    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-maas-api-tls\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.659395 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.659120    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-metrics-certs\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.659395 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.659204    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e4d998-cc95-4e59-9192-07bc8417c31c-openshift-service-ca\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.760220 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.760103    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b92nf\" (UniqueName: \"kubernetes.io/projected/44e4d998-cc95-4e59-9192-07bc8417c31c-kube-api-access-b92nf\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.760220 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.760171    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-maas-api-tls\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.760220 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.760208    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-metrics-certs\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.760555 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.760243    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e4d998-cc95-4e59-9192-07bc8417c31c-openshift-service-ca\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.760555 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:19.760387    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-314824-serving-cert: secret "maas-api-e2e-autores-314824-serving-cert" not found
Sep 10 15:55:19.760555 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:19.760458    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-maas-api-tls podName:44e4d998-cc95-4e59-9192-07bc8417c31c nodeName:}" failed. No retries permitted until 2026-09-10 15:55:20.260440657 +0000 UTC m=+816.176565515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-maas-api-tls") pod "maas-api-e2e-autores-314824-7588dc75b9-kx7h2" (UID: "44e4d998-cc95-4e59-9192-07bc8417c31c") : secret "maas-api-e2e-autores-314824-serving-cert" not found
Sep 10 15:55:19.761075 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.761052    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e4d998-cc95-4e59-9192-07bc8417c31c-openshift-service-ca\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.763326 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.763304    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-metrics-certs\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:19.769845 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:19.769818    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b92nf\" (UniqueName: \"kubernetes.io/projected/44e4d998-cc95-4e59-9192-07bc8417c31c-kube-api-access-b92nf\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:20.242855 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.242815    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"]
Sep 10 15:55:20.246842 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.246824    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:20.249164 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.249121    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-314824-dockercfg-4krws\""
Sep 10 15:55:20.249357 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.249195    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-314824\""
Sep 10 15:55:20.256525 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.255991    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"]
Sep 10 15:55:20.264935 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.264878    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-maas-api-tls\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:20.267689 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.267660    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-maas-api-tls\") pod \"maas-api-e2e-autores-314824-7588dc75b9-kx7h2\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:20.365706 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.365674    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e5665f4-3d83-407a-b63d-7c38bb26dd70-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v\" (UID: \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:20.365897 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.365747    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjvw6\" (UniqueName: \"kubernetes.io/projected/8e5665f4-3d83-407a-b63d-7c38bb26dd70-kube-api-access-hjvw6\") pod \"payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v\" (UID: \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:20.409558 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.409515    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:20.467121 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.467094    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjvw6\" (UniqueName: \"kubernetes.io/projected/8e5665f4-3d83-407a-b63d-7c38bb26dd70-kube-api-access-hjvw6\") pod \"payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v\" (UID: \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:20.467294 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.467217    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e5665f4-3d83-407a-b63d-7c38bb26dd70-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v\" (UID: \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:20.468098 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.468074    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e5665f4-3d83-407a-b63d-7c38bb26dd70-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v\" (UID: \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:20.477944 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.477877    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjvw6\" (UniqueName: \"kubernetes.io/projected/8e5665f4-3d83-407a-b63d-7c38bb26dd70-kube-api-access-hjvw6\") pod \"payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v\" (UID: \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:20.559932 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.559888    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:20.689341 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.689312    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"]
Sep 10 15:55:20.691245 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:55:20.691215    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e4d998_cc95_4e59_9192_07bc8417c31c.slice/crio-0c97946113281847d88145fc96403d8e421d06b98fbd7969978490cf19095e81 WatchSource:0}: Error finding container 0c97946113281847d88145fc96403d8e421d06b98fbd7969978490cf19095e81: Status 404 returned error can't find the container with id 0c97946113281847d88145fc96403d8e421d06b98fbd7969978490cf19095e81
Sep 10 15:55:20.839728 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.839646    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"]
Sep 10 15:55:20.842616 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:55:20.842581    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e5665f4_3d83_407a_b63d_7c38bb26dd70.slice/crio-277e313ae154903a211ea49c167fee1bd7060a1dd561fc0fb28c2b3e6ac2e272 WatchSource:0}: Error finding container 277e313ae154903a211ea49c167fee1bd7060a1dd561fc0fb28c2b3e6ac2e272: Status 404 returned error can't find the container with id 277e313ae154903a211ea49c167fee1bd7060a1dd561fc0fb28c2b3e6ac2e272
Sep 10 15:55:20.843629 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:20.843605    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2" event={"ID":"44e4d998-cc95-4e59-9192-07bc8417c31c","Type":"ContainerStarted","Data":"0c97946113281847d88145fc96403d8e421d06b98fbd7969978490cf19095e81"}
Sep 10 15:55:21.854825 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:21.854574    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2" event={"ID":"44e4d998-cc95-4e59-9192-07bc8417c31c","Type":"ContainerStarted","Data":"7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea"}
Sep 10 15:55:21.854825 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:21.854760    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:21.856994 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:21.856962    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v" event={"ID":"8e5665f4-3d83-407a-b63d-7c38bb26dd70","Type":"ContainerStarted","Data":"3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f"}
Sep 10 15:55:21.857211 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:21.857017    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v" event={"ID":"8e5665f4-3d83-407a-b63d-7c38bb26dd70","Type":"ContainerStarted","Data":"277e313ae154903a211ea49c167fee1bd7060a1dd561fc0fb28c2b3e6ac2e272"}
Sep 10 15:55:21.857211 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:21.857049    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:21.876505 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:21.876441    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2" podStartSLOduration=1.895612646 podStartE2EDuration="2.876419182s" totalImagesPullingTime="980.806536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:55:21.874034205 +0000 UTC m=+817.790159107" watchObservedRunningTime="2026-09-10 15:55:21.876419182 +0000 UTC m=+817.792544112"
Sep 10 15:55:21.895974 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:21.895906    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v" podStartSLOduration=1.895887236 podStartE2EDuration="1.895887236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:55:21.895111194 +0000 UTC m=+817.811236095" watchObservedRunningTime="2026-09-10 15:55:21.895887236 +0000 UTC m=+817.812012114"
Sep 10 15:55:23.832701 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:23.832670    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:27.869511 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:27.869477    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:55:28.829325 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:28.829294    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:32.866063 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:32.866027    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:55:37.606889 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.606849    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"]
Sep 10 15:55:37.612381 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.612347    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.615069 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.615043    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-5d6f34-serving-cert\""
Sep 10 15:55:37.626345 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.626315    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"]
Sep 10 15:55:37.731234 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.731191    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.731418 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.731255    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-maas-api-tls\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.731418 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.731350    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-metrics-certs\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.731418 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.731402    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8pnq\" (UniqueName: \"kubernetes.io/projected/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-kube-api-access-g8pnq\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.832068 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.832020    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-metrics-certs\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.832324 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.832084    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8pnq\" (UniqueName: \"kubernetes.io/projected/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-kube-api-access-g8pnq\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.832324 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.832190    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.832324 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.832215    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-maas-api-tls\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.832485 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:37.832336    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-serving-cert: secret "maas-api-e2e-inf-b-5d6f34-serving-cert" not found
Sep 10 15:55:37.832485 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:37.832406    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-maas-api-tls podName:bc5a112b-9a7c-4326-a00c-17b214f4a5fe nodeName:}" failed. No retries permitted until 2026-09-10 15:55:38.332384868 +0000 UTC m=+834.248509745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-maas-api-tls") pod "maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk" (UID: "bc5a112b-9a7c-4326-a00c-17b214f4a5fe") : secret "maas-api-e2e-inf-b-5d6f34-serving-cert" not found
Sep 10 15:55:37.833016 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.832988    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.834738 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.834711    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-metrics-certs\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:37.841425 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:37.841399    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8pnq\" (UniqueName: \"kubernetes.io/projected/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-kube-api-access-g8pnq\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:38.303649 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.303605    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"]
Sep 10 15:55:38.307690 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.307663    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:38.310291 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.310267    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5d6f34\""
Sep 10 15:55:38.319796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.319764    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"]
Sep 10 15:55:38.337807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.337761    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-maas-api-tls\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:38.340638 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.340606    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-maas-api-tls\") pod \"maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:38.439161 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.439101    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htfjg\" (UniqueName: \"kubernetes.io/projected/dda3fd75-5ec6-49f7-885f-9656380b924f-kube-api-access-htfjg\") pod \"payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22\" (UID: \"dda3fd75-5ec6-49f7-885f-9656380b924f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:38.439375 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.439250    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda3fd75-5ec6-49f7-885f-9656380b924f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22\" (UID: \"dda3fd75-5ec6-49f7-885f-9656380b924f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:38.525243 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.525198    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:38.540419 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.540376    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htfjg\" (UniqueName: \"kubernetes.io/projected/dda3fd75-5ec6-49f7-885f-9656380b924f-kube-api-access-htfjg\") pod \"payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22\" (UID: \"dda3fd75-5ec6-49f7-885f-9656380b924f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:38.540666 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.540485    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda3fd75-5ec6-49f7-885f-9656380b924f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22\" (UID: \"dda3fd75-5ec6-49f7-885f-9656380b924f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:38.541463 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.541436    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda3fd75-5ec6-49f7-885f-9656380b924f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22\" (UID: \"dda3fd75-5ec6-49f7-885f-9656380b924f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:38.551257 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.551228    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htfjg\" (UniqueName: \"kubernetes.io/projected/dda3fd75-5ec6-49f7-885f-9656380b924f-kube-api-access-htfjg\") pod \"payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22\" (UID: \"dda3fd75-5ec6-49f7-885f-9656380b924f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:38.620650 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.620612    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:38.809747 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.809669    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"]
Sep 10 15:55:38.812865 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:55:38.812822    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5a112b_9a7c_4326_a00c_17b214f4a5fe.slice/crio-22929957946f19a232bd982e7a37a6e58c036a841fd570e9eaf3762fdea4fc34 WatchSource:0}: Error finding container 22929957946f19a232bd982e7a37a6e58c036a841fd570e9eaf3762fdea4fc34: Status 404 returned error can't find the container with id 22929957946f19a232bd982e7a37a6e58c036a841fd570e9eaf3762fdea4fc34
Sep 10 15:55:38.898448 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.898419    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"]
Sep 10 15:55:38.901285 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:55:38.901259    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda3fd75_5ec6_49f7_885f_9656380b924f.slice/crio-77040f36c101fd165007bc48079d5d79c7c3ecca3033fcfe51d56033b3ee1fc2 WatchSource:0}: Error finding container 77040f36c101fd165007bc48079d5d79c7c3ecca3033fcfe51d56033b3ee1fc2: Status 404 returned error can't find the container with id 77040f36c101fd165007bc48079d5d79c7c3ecca3033fcfe51d56033b3ee1fc2
Sep 10 15:55:38.957502 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.957467    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22" event={"ID":"dda3fd75-5ec6-49f7-885f-9656380b924f","Type":"ContainerStarted","Data":"77040f36c101fd165007bc48079d5d79c7c3ecca3033fcfe51d56033b3ee1fc2"}
Sep 10 15:55:38.959162 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:38.959105    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk" event={"ID":"bc5a112b-9a7c-4326-a00c-17b214f4a5fe","Type":"ContainerStarted","Data":"22929957946f19a232bd982e7a37a6e58c036a841fd570e9eaf3762fdea4fc34"}
Sep 10 15:55:39.967148 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:39.967092    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk" event={"ID":"bc5a112b-9a7c-4326-a00c-17b214f4a5fe","Type":"ContainerStarted","Data":"c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e"}
Sep 10 15:55:39.967634 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:39.967296    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:39.968940 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:39.968911    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22" event={"ID":"dda3fd75-5ec6-49f7-885f-9656380b924f","Type":"ContainerStarted","Data":"29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13"}
Sep 10 15:55:39.969058 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:39.969051    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:39.985758 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:39.985703    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk" podStartSLOduration=2.514568593 podStartE2EDuration="2.985684525s" totalImagesPullingTime="471.115932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:55:39.985220635 +0000 UTC m=+835.901345527" watchObservedRunningTime="2026-09-10 15:55:39.985684525 +0000 UTC m=+835.901809442"
Sep 10 15:55:40.003275 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:40.003220    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22" podStartSLOduration=2.00320394 podStartE2EDuration="2.00320394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:55:40.001446948 +0000 UTC m=+835.917571847" watchObservedRunningTime="2026-09-10 15:55:40.00320394 +0000 UTC m=+835.919328819"
Sep 10 15:55:45.981464 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:45.981431    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:55:50.976664 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:50.976628    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:55:56.478802 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:56.478762    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"]
Sep 10 15:55:56.479360 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:56.479017    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx" podUID="3d94a7a2-71fb-4aff-a945-f0d9e58824c2" containerName="maas-api" containerID="cri-o://fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e" gracePeriod=30
Sep 10 15:55:56.569282 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:56.569245    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"]
Sep 10 15:55:56.569805 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:56.569744    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h" podUID="368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e" containerName="payload-pre-processing" containerID="cri-o://3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521" gracePeriod=30
Sep 10 15:55:56.874453 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:56.874413    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:56.877382 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:56.877359    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:57.022549 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.022439    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m799d\" (UniqueName: \"kubernetes.io/projected/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-kube-api-access-m799d\") pod \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") "
Sep 10 15:55:57.022549 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.022505    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-openshift-service-ca\") pod \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") "
Sep 10 15:55:57.022549 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.022536    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-metrics-certs\") pod \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") "
Sep 10 15:55:57.022549 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.022562    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwf98\" (UniqueName: \"kubernetes.io/projected/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-kube-api-access-cwf98\") pod \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\" (UID: \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\") "
Sep 10 15:55:57.022908 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.022611    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-maas-api-tls\") pod \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\" (UID: \"3d94a7a2-71fb-4aff-a945-f0d9e58824c2\") "
Sep 10 15:55:57.022908 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.022644    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-plugins-config-volume\") pod \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\" (UID: \"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e\") "
Sep 10 15:55:57.023068 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.022950    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d94a7a2-71fb-4aff-a945-f0d9e58824c2" (UID: "3d94a7a2-71fb-4aff-a945-f0d9e58824c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:55:57.023188 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.023164    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e" (UID: "368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:55:57.024999 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.024968    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-kube-api-access-cwf98" (OuterVolumeSpecName: "kube-api-access-cwf98") pod "368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e" (UID: "368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e"). InnerVolumeSpecName "kube-api-access-cwf98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:55:57.024999 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.024981    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3d94a7a2-71fb-4aff-a945-f0d9e58824c2" (UID: "3d94a7a2-71fb-4aff-a945-f0d9e58824c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:55:57.025231 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.025097    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-kube-api-access-m799d" (OuterVolumeSpecName: "kube-api-access-m799d") pod "3d94a7a2-71fb-4aff-a945-f0d9e58824c2" (UID: "3d94a7a2-71fb-4aff-a945-f0d9e58824c2"). InnerVolumeSpecName "kube-api-access-m799d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:55:57.025231 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.025125    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d94a7a2-71fb-4aff-a945-f0d9e58824c2" (UID: "3d94a7a2-71fb-4aff-a945-f0d9e58824c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:55:57.065372 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.065329    2527 generic.go:356] "Generic (PLEG): container finished" podID="3d94a7a2-71fb-4aff-a945-f0d9e58824c2" containerID="fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e" exitCode=0
Sep 10 15:55:57.065372 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.065375    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"
Sep 10 15:55:57.065614 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.065408    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx" event={"ID":"3d94a7a2-71fb-4aff-a945-f0d9e58824c2","Type":"ContainerDied","Data":"fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e"}
Sep 10 15:55:57.065614 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.065461    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx" event={"ID":"3d94a7a2-71fb-4aff-a945-f0d9e58824c2","Type":"ContainerDied","Data":"dc2dc34029493fb0ac77fd27203622609c8ece84845abab1fb75a30f82215965"}
Sep 10 15:55:57.065614 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.065485    2527 scope.go:117] "RemoveContainer" containerID="fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e"
Sep 10 15:55:57.066962 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.066936    2527 generic.go:356] "Generic (PLEG): container finished" podID="368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e" containerID="3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521" exitCode=0
Sep 10 15:55:57.067077 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.066969    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h" event={"ID":"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e","Type":"ContainerDied","Data":"3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521"}
Sep 10 15:55:57.067077 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.067010    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h" event={"ID":"368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e","Type":"ContainerDied","Data":"483291aeea2aa08bf46631042543da8aa99353933e47d1f7f72e25607308f5e8"}
Sep 10 15:55:57.067077 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.066977    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"
Sep 10 15:55:57.075940 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.075915    2527 scope.go:117] "RemoveContainer" containerID="fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e"
Sep 10 15:55:57.076316 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:57.076294    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e\": container with ID starting with fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e not found: ID does not exist" containerID="fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e"
Sep 10 15:55:57.076384 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.076346    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e"} err="failed to get container status \"fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e\": rpc error: code = NotFound desc = could not find container \"fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e\": container with ID starting with fd070101511367b2816d8295fdd0b019f4854019b7eb700e9785b2a750291d5e not found: ID does not exist"
Sep 10 15:55:57.076384 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.076366    2527 scope.go:117] "RemoveContainer" containerID="3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521"
Sep 10 15:55:57.085501 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.085478    2527 scope.go:117] "RemoveContainer" containerID="3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521"
Sep 10 15:55:57.085806 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:55:57.085784    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521\": container with ID starting with 3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521 not found: ID does not exist" containerID="3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521"
Sep 10 15:55:57.085870 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.085813    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521"} err="failed to get container status \"3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521\": rpc error: code = NotFound desc = could not find container \"3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521\": container with ID starting with 3246549a9df3520c2d8709cc2875178283f99e8c13bfb81068b7c712bdcf7521 not found: ID does not exist"
Sep 10 15:55:57.091546 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.091517    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"]
Sep 10 15:55:57.095667 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.095641    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-941c6070-64bbdb7d9d-kr9kx"]
Sep 10 15:55:57.109154 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.109095    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"]
Sep 10 15:55:57.117191 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.117159    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-941c6070-595f5fc8b-zth5h"]
Sep 10 15:55:57.124435 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.124406    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:55:57.124541 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.124440    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:55:57.124541 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.124454    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwf98\" (UniqueName: \"kubernetes.io/projected/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-kube-api-access-cwf98\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:55:57.124541 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.124463    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:55:57.124541 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.124472    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:55:57.124541 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:57.124480    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m799d\" (UniqueName: \"kubernetes.io/projected/3d94a7a2-71fb-4aff-a945-f0d9e58824c2-kube-api-access-m799d\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:55:58.564125 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:58.564078    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e" path="/var/lib/kubelet/pods/368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e/volumes"
Sep 10 15:55:58.564718 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:55:58.564693    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d94a7a2-71fb-4aff-a945-f0d9e58824c2" path="/var/lib/kubelet/pods/3d94a7a2-71fb-4aff-a945-f0d9e58824c2/volumes"
Sep 10 15:56:00.716086 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.716040    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"]
Sep 10 15:56:00.717063 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.717013    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d94a7a2-71fb-4aff-a945-f0d9e58824c2" containerName="maas-api"
Sep 10 15:56:00.717063 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.717044    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d94a7a2-71fb-4aff-a945-f0d9e58824c2" containerName="maas-api"
Sep 10 15:56:00.717063 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.717060    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e" containerName="payload-pre-processing"
Sep 10 15:56:00.717063 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.717068    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e" containerName="payload-pre-processing"
Sep 10 15:56:00.717427 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.717261    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d94a7a2-71fb-4aff-a945-f0d9e58824c2" containerName="maas-api"
Sep 10 15:56:00.717427 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.717279    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="368ab3ac-d0cc-49bf-b82d-d862fc5f8c0e" containerName="payload-pre-processing"
Sep 10 15:56:00.722228 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.722195    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.729526 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.729491    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-0fcd7b0f-serving-cert\""
Sep 10 15:56:00.736699 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.736664    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"]
Sep 10 15:56:00.860637 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.860592    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjxvv\" (UniqueName: \"kubernetes.io/projected/20157513-5344-4592-8c8a-ff6645b62164-kube-api-access-sjxvv\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.860637 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.860641    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-maas-api-tls\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.860875 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.860692    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20157513-5344-4592-8c8a-ff6645b62164-openshift-service-ca\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.860875 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.860791    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-metrics-certs\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.962310 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.962266    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjxvv\" (UniqueName: \"kubernetes.io/projected/20157513-5344-4592-8c8a-ff6645b62164-kube-api-access-sjxvv\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.962310 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.962314    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-maas-api-tls\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.962579 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.962334    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20157513-5344-4592-8c8a-ff6645b62164-openshift-service-ca\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.962579 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.962389    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-metrics-certs\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.963082 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.963055    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20157513-5344-4592-8c8a-ff6645b62164-openshift-service-ca\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.964817 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.964795    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-maas-api-tls\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.964931 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.964913    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-metrics-certs\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:00.970861 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:00.970799    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjxvv\" (UniqueName: \"kubernetes.io/projected/20157513-5344-4592-8c8a-ff6645b62164-kube-api-access-sjxvv\") pod \"maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:01.034920 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.034873    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:01.322683 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.322656    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"]
Sep 10 15:56:01.325396 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:56:01.325354    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20157513_5344_4592_8c8a_ff6645b62164.slice/crio-33ab3fe9006087d149fc02b97e74a468a415787a760a8a035f4201f25383580a WatchSource:0}: Error finding container 33ab3fe9006087d149fc02b97e74a468a415787a760a8a035f4201f25383580a: Status 404 returned error can't find the container with id 33ab3fe9006087d149fc02b97e74a468a415787a760a8a035f4201f25383580a
Sep 10 15:56:01.393158 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.393105    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"]
Sep 10 15:56:01.398062 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.398030    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:01.401529 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.401504    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-0fcd7b0f\""
Sep 10 15:56:01.401686 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.401566    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-0fcd7b0f-dockercfg-qh5m2\""
Sep 10 15:56:01.412028 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.411998    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"]
Sep 10 15:56:01.567742 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.567698    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb\" (UID: \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\") " pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:01.567946 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.567753    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6bjp\" (UniqueName: \"kubernetes.io/projected/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-kube-api-access-r6bjp\") pod \"payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb\" (UID: \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\") " pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:01.669182 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.669042    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb\" (UID: \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\") " pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:01.669182 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.669090    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6bjp\" (UniqueName: \"kubernetes.io/projected/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-kube-api-access-r6bjp\") pod \"payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb\" (UID: \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\") " pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:01.669916 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.669891    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb\" (UID: \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\") " pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:01.677627 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.677596    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6bjp\" (UniqueName: \"kubernetes.io/projected/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-kube-api-access-r6bjp\") pod \"payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb\" (UID: \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\") " pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:01.710563 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.710520    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:01.995987 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:01.995959    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"]
Sep 10 15:56:01.996909 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:56:01.996868    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ed8bd1_c48d_4747_bae3_2fb1859f6f33.slice/crio-422818f341f80cdb1b6b3ed5123bceb5f80bc8ccaf926735c202b0c587c2ef28 WatchSource:0}: Error finding container 422818f341f80cdb1b6b3ed5123bceb5f80bc8ccaf926735c202b0c587c2ef28: Status 404 returned error can't find the container with id 422818f341f80cdb1b6b3ed5123bceb5f80bc8ccaf926735c202b0c587c2ef28
Sep 10 15:56:02.100513 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:02.100457    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq" event={"ID":"20157513-5344-4592-8c8a-ff6645b62164","Type":"ContainerStarted","Data":"333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177"}
Sep 10 15:56:02.100711 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:02.100537    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:02.100711 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:02.100563    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq" event={"ID":"20157513-5344-4592-8c8a-ff6645b62164","Type":"ContainerStarted","Data":"33ab3fe9006087d149fc02b97e74a468a415787a760a8a035f4201f25383580a"}
Sep 10 15:56:02.102463 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:02.102434    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb" event={"ID":"77ed8bd1-c48d-4747-bae3-2fb1859f6f33","Type":"ContainerStarted","Data":"898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164"}
Sep 10 15:56:02.102601 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:02.102485    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb" event={"ID":"77ed8bd1-c48d-4747-bae3-2fb1859f6f33","Type":"ContainerStarted","Data":"422818f341f80cdb1b6b3ed5123bceb5f80bc8ccaf926735c202b0c587c2ef28"}
Sep 10 15:56:02.102601 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:02.102519    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:02.118434 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:02.118376    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq" podStartSLOduration=1.519227716 podStartE2EDuration="2.118355962s" totalImagesPullingTime="599.128246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:02.118062154 +0000 UTC m=+858.034187053" watchObservedRunningTime="2026-09-10 15:56:02.118355962 +0000 UTC m=+858.034480841"
Sep 10 15:56:02.137246 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:02.137171    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb" podStartSLOduration=1.137108023 podStartE2EDuration="1.137108023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:02.135049027 +0000 UTC m=+858.051173919" watchObservedRunningTime="2026-09-10 15:56:02.137108023 +0000 UTC m=+858.053232903"
Sep 10 15:56:08.113787 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:08.113753    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:13.111192 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:13.111159    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:41.907387 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:41.907283    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"]
Sep 10 15:56:41.907875 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:41.907608    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq" podUID="20157513-5344-4592-8c8a-ff6645b62164" containerName="maas-api" containerID="cri-o://333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177" gracePeriod=30
Sep 10 15:56:41.960024 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:41.959984    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"]
Sep 10 15:56:41.960344 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:41.960280    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb" podUID="77ed8bd1-c48d-4747-bae3-2fb1859f6f33" containerName="payload-processing" containerID="cri-o://898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164" gracePeriod=30
Sep 10 15:56:42.221212 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.221185    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:42.252057 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.252023    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:42.323327 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.323291    2527 generic.go:356] "Generic (PLEG): container finished" podID="20157513-5344-4592-8c8a-ff6645b62164" containerID="333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177" exitCode=0
Sep 10 15:56:42.323508 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.323340    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"
Sep 10 15:56:42.323508 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.323336    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq" event={"ID":"20157513-5344-4592-8c8a-ff6645b62164","Type":"ContainerDied","Data":"333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177"}
Sep 10 15:56:42.323508 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.323462    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq" event={"ID":"20157513-5344-4592-8c8a-ff6645b62164","Type":"ContainerDied","Data":"33ab3fe9006087d149fc02b97e74a468a415787a760a8a035f4201f25383580a"}
Sep 10 15:56:42.323508 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.323487    2527 scope.go:117] "RemoveContainer" containerID="333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177"
Sep 10 15:56:42.325026 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.325003    2527 generic.go:356] "Generic (PLEG): container finished" podID="77ed8bd1-c48d-4747-bae3-2fb1859f6f33" containerID="898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164" exitCode=0
Sep 10 15:56:42.325126 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.325046    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb" event={"ID":"77ed8bd1-c48d-4747-bae3-2fb1859f6f33","Type":"ContainerDied","Data":"898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164"}
Sep 10 15:56:42.325126 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.325057    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"
Sep 10 15:56:42.325126 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.325081    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb" event={"ID":"77ed8bd1-c48d-4747-bae3-2fb1859f6f33","Type":"ContainerDied","Data":"422818f341f80cdb1b6b3ed5123bceb5f80bc8ccaf926735c202b0c587c2ef28"}
Sep 10 15:56:42.333826 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.333796    2527 scope.go:117] "RemoveContainer" containerID="333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177"
Sep 10 15:56:42.334184 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:56:42.334163    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177\": container with ID starting with 333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177 not found: ID does not exist" containerID="333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177"
Sep 10 15:56:42.334258 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.334195    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177"} err="failed to get container status \"333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177\": rpc error: code = NotFound desc = could not find container \"333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177\": container with ID starting with 333d400d0ccd18d5534a2b76c2a7b15f1f98cbcb929e103979c84341a193c177 not found: ID does not exist"
Sep 10 15:56:42.334258 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.334215    2527 scope.go:117] "RemoveContainer" containerID="898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164"
Sep 10 15:56:42.345705 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.345679    2527 scope.go:117] "RemoveContainer" containerID="898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164"
Sep 10 15:56:42.346120 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:56:42.346099    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164\": container with ID starting with 898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164 not found: ID does not exist" containerID="898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164"
Sep 10 15:56:42.346202 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.346147    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164"} err="failed to get container status \"898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164\": rpc error: code = NotFound desc = could not find container \"898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164\": container with ID starting with 898cd682c03e93f1af7aeb4c1abc843c29c4666709b50b8534492da484e92164 not found: ID does not exist"
Sep 10 15:56:42.369540 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.369500    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-plugins-config-volume\") pod \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\" (UID: \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\") "
Sep 10 15:56:42.369540 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.369547    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6bjp\" (UniqueName: \"kubernetes.io/projected/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-kube-api-access-r6bjp\") pod \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\" (UID: \"77ed8bd1-c48d-4747-bae3-2fb1859f6f33\") "
Sep 10 15:56:42.369849 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.369591    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20157513-5344-4592-8c8a-ff6645b62164-openshift-service-ca\") pod \"20157513-5344-4592-8c8a-ff6645b62164\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") "
Sep 10 15:56:42.369849 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.369632    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjxvv\" (UniqueName: \"kubernetes.io/projected/20157513-5344-4592-8c8a-ff6645b62164-kube-api-access-sjxvv\") pod \"20157513-5344-4592-8c8a-ff6645b62164\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") "
Sep 10 15:56:42.369849 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.369660    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-maas-api-tls\") pod \"20157513-5344-4592-8c8a-ff6645b62164\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") "
Sep 10 15:56:42.369849 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.369698    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-metrics-certs\") pod \"20157513-5344-4592-8c8a-ff6645b62164\" (UID: \"20157513-5344-4592-8c8a-ff6645b62164\") "
Sep 10 15:56:42.370058 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.370009    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "77ed8bd1-c48d-4747-bae3-2fb1859f6f33" (UID: "77ed8bd1-c48d-4747-bae3-2fb1859f6f33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:56:42.370058 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.370035    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20157513-5344-4592-8c8a-ff6645b62164-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20157513-5344-4592-8c8a-ff6645b62164" (UID: "20157513-5344-4592-8c8a-ff6645b62164"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:56:42.372067 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.372026    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20157513-5344-4592-8c8a-ff6645b62164-kube-api-access-sjxvv" (OuterVolumeSpecName: "kube-api-access-sjxvv") pod "20157513-5344-4592-8c8a-ff6645b62164" (UID: "20157513-5344-4592-8c8a-ff6645b62164"). InnerVolumeSpecName "kube-api-access-sjxvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:56:42.372067 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.372057    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "20157513-5344-4592-8c8a-ff6645b62164" (UID: "20157513-5344-4592-8c8a-ff6645b62164"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:56:42.372243 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.372073    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-kube-api-access-r6bjp" (OuterVolumeSpecName: "kube-api-access-r6bjp") pod "77ed8bd1-c48d-4747-bae3-2fb1859f6f33" (UID: "77ed8bd1-c48d-4747-bae3-2fb1859f6f33"). InnerVolumeSpecName "kube-api-access-r6bjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:56:42.372243 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.372082    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20157513-5344-4592-8c8a-ff6645b62164" (UID: "20157513-5344-4592-8c8a-ff6645b62164"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:56:42.470665 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.470554    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjxvv\" (UniqueName: \"kubernetes.io/projected/20157513-5344-4592-8c8a-ff6645b62164-kube-api-access-sjxvv\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:42.470665 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.470589    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:42.470665 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.470604    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20157513-5344-4592-8c8a-ff6645b62164-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:42.470665 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.470615    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:42.470665 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.470626    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6bjp\" (UniqueName: \"kubernetes.io/projected/77ed8bd1-c48d-4747-bae3-2fb1859f6f33-kube-api-access-r6bjp\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:42.470665 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.470638    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20157513-5344-4592-8c8a-ff6645b62164-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:42.659311 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.659273    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"]
Sep 10 15:56:42.676945 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.676904    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0fcd7b0f-6c8d9f479c-b88kq"]
Sep 10 15:56:42.694435 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.694387    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"]
Sep 10 15:56:42.708064 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:42.708024    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0fcd7b0f-6b6c98b58-hw7pb"]
Sep 10 15:56:44.551995 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:44.551966    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 15:56:44.552841 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:44.552825    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 15:56:44.572701 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:44.572661    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20157513-5344-4592-8c8a-ff6645b62164" path="/var/lib/kubelet/pods/20157513-5344-4592-8c8a-ff6645b62164/volumes"
Sep 10 15:56:44.573124 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:44.573106    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77ed8bd1-c48d-4747-bae3-2fb1859f6f33" path="/var/lib/kubelet/pods/77ed8bd1-c48d-4747-bae3-2fb1859f6f33/volumes"
Sep 10 15:56:47.018625 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.018585    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"]
Sep 10 15:56:47.019122 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.019104    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77ed8bd1-c48d-4747-bae3-2fb1859f6f33" containerName="payload-processing"
Sep 10 15:56:47.019223 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.019123    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="77ed8bd1-c48d-4747-bae3-2fb1859f6f33" containerName="payload-processing"
Sep 10 15:56:47.019223 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.019185    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20157513-5344-4592-8c8a-ff6645b62164" containerName="maas-api"
Sep 10 15:56:47.019223 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.019192    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="20157513-5344-4592-8c8a-ff6645b62164" containerName="maas-api"
Sep 10 15:56:47.019372 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.019263    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="77ed8bd1-c48d-4747-bae3-2fb1859f6f33" containerName="payload-processing"
Sep 10 15:56:47.019372 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.019270    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="20157513-5344-4592-8c8a-ff6645b62164" containerName="maas-api"
Sep 10 15:56:47.024493 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.024468    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.027671 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.027637    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-529c3efe-serving-cert\""
Sep 10 15:56:47.098470 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.098433    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"]
Sep 10 15:56:47.109517 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.109477    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-maas-api-tls\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.109717 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.109559    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bc2ceb-6487-4314-8323-d5aad8436dd0-openshift-service-ca\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.109717 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.109611    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfkr8\" (UniqueName: \"kubernetes.io/projected/72bc2ceb-6487-4314-8323-d5aad8436dd0-kube-api-access-hfkr8\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.109717 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.109704    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-metrics-certs\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.210222 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.210182    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-metrics-certs\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.210444 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.210246    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-maas-api-tls\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.210444 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.210293    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bc2ceb-6487-4314-8323-d5aad8436dd0-openshift-service-ca\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.210578 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.210437    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfkr8\" (UniqueName: \"kubernetes.io/projected/72bc2ceb-6487-4314-8323-d5aad8436dd0-kube-api-access-hfkr8\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.210989 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.210968    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bc2ceb-6487-4314-8323-d5aad8436dd0-openshift-service-ca\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.212937 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.212914    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-metrics-certs\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.213061 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.212950    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-maas-api-tls\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.225343 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.225308    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfkr8\" (UniqueName: \"kubernetes.io/projected/72bc2ceb-6487-4314-8323-d5aad8436dd0-kube-api-access-hfkr8\") pod \"maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.335905 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.335864    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:47.617478 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.617365    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"]
Sep 10 15:56:47.620797 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:56:47.620768    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bc2ceb_6487_4314_8323_d5aad8436dd0.slice/crio-235dadbb84eb4619b36478795f06bce1c79b436ae507423e60864719f561a15f WatchSource:0}: Error finding container 235dadbb84eb4619b36478795f06bce1c79b436ae507423e60864719f561a15f: Status 404 returned error can't find the container with id 235dadbb84eb4619b36478795f06bce1c79b436ae507423e60864719f561a15f
Sep 10 15:56:47.789238 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.789197    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"]
Sep 10 15:56:47.794143 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.794107    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:56:47.799336 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.799311    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-529c3efe\""
Sep 10 15:56:47.799503 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.799450    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-529c3efe-dockercfg-g2xqq\""
Sep 10 15:56:47.807676 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.807643    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"]
Sep 10 15:56:47.815934 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.815899    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5mg7\" (UniqueName: \"kubernetes.io/projected/a5cba170-19bf-4cbe-adea-8eb7ecf84466-kube-api-access-z5mg7\") pod \"payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg\" (UID: \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\") " pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:56:47.816111 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.816030    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5cba170-19bf-4cbe-adea-8eb7ecf84466-plugins-config-volume\") pod \"payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg\" (UID: \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\") " pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:56:47.917126 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.917031    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5cba170-19bf-4cbe-adea-8eb7ecf84466-plugins-config-volume\") pod \"payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg\" (UID: \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\") " pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:56:47.917126 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.917087    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5mg7\" (UniqueName: \"kubernetes.io/projected/a5cba170-19bf-4cbe-adea-8eb7ecf84466-kube-api-access-z5mg7\") pod \"payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg\" (UID: \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\") " pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:56:47.917866 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.917835    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5cba170-19bf-4cbe-adea-8eb7ecf84466-plugins-config-volume\") pod \"payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg\" (UID: \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\") " pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:56:47.929666 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:47.929640    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5mg7\" (UniqueName: \"kubernetes.io/projected/a5cba170-19bf-4cbe-adea-8eb7ecf84466-kube-api-access-z5mg7\") pod \"payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg\" (UID: \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\") " pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:56:48.108913 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:48.108883    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:56:48.364104 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:48.364056    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9" event={"ID":"72bc2ceb-6487-4314-8323-d5aad8436dd0","Type":"ContainerStarted","Data":"859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80"}
Sep 10 15:56:48.364318 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:48.364193    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9" event={"ID":"72bc2ceb-6487-4314-8323-d5aad8436dd0","Type":"ContainerStarted","Data":"235dadbb84eb4619b36478795f06bce1c79b436ae507423e60864719f561a15f"}
Sep 10 15:56:48.364379 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:48.364322    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:48.382263 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:48.382206    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9" podStartSLOduration=1.923184836 podStartE2EDuration="2.382186768s" totalImagesPullingTime="459.001932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:48.380629088 +0000 UTC m=+904.296754005" watchObservedRunningTime="2026-09-10 15:56:48.382186768 +0000 UTC m=+904.298311648"
Sep 10 15:56:48.394762 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:48.394736    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"]
Sep 10 15:56:48.397037 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:56:48.396991    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5cba170_19bf_4cbe_adea_8eb7ecf84466.slice/crio-b65753013358b6a51b25d4e4af943e0af033bc997c3adad47c202d57f53a4651 WatchSource:0}: Error finding container b65753013358b6a51b25d4e4af943e0af033bc997c3adad47c202d57f53a4651: Status 404 returned error can't find the container with id b65753013358b6a51b25d4e4af943e0af033bc997c3adad47c202d57f53a4651
Sep 10 15:56:49.372372 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:49.372329    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg" event={"ID":"a5cba170-19bf-4cbe-adea-8eb7ecf84466","Type":"ContainerStarted","Data":"dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c"}
Sep 10 15:56:49.372804 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:49.372391    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg" event={"ID":"a5cba170-19bf-4cbe-adea-8eb7ecf84466","Type":"ContainerStarted","Data":"b65753013358b6a51b25d4e4af943e0af033bc997c3adad47c202d57f53a4651"}
Sep 10 15:56:49.372804 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:49.372687    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:56:49.387235 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:49.387161    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg" podStartSLOduration=2.387116539 podStartE2EDuration="2.387116539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:56:49.386732372 +0000 UTC m=+905.302857275" watchObservedRunningTime="2026-09-10 15:56:49.387116539 +0000 UTC m=+905.303241417"
Sep 10 15:56:52.634096 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:52.634054    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"]
Sep 10 15:56:52.634556 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:52.634342    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2" podUID="44e4d998-cc95-4e59-9192-07bc8417c31c" containerName="maas-api" containerID="cri-o://7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea" gracePeriod=30
Sep 10 15:56:52.696340 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:52.696301    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"]
Sep 10 15:56:52.696615 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:52.696563    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v" podUID="8e5665f4-3d83-407a-b63d-7c38bb26dd70" containerName="payload-pre-processing" containerID="cri-o://3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f" gracePeriod=30
Sep 10 15:56:52.864504 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:52.864467    2527 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v" podUID="8e5665f4-3d83-407a-b63d-7c38bb26dd70" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.47:9004: connect: connection refused"
Sep 10 15:56:52.971344 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:52.971307    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:56:53.005896 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.005865    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:56:53.065317 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.065270    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-maas-api-tls\") pod \"44e4d998-cc95-4e59-9192-07bc8417c31c\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") "
Sep 10 15:56:53.065535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.065329    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e5665f4-3d83-407a-b63d-7c38bb26dd70-plugins-config-volume\") pod \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\" (UID: \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\") "
Sep 10 15:56:53.065535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.065374    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjvw6\" (UniqueName: \"kubernetes.io/projected/8e5665f4-3d83-407a-b63d-7c38bb26dd70-kube-api-access-hjvw6\") pod \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\" (UID: \"8e5665f4-3d83-407a-b63d-7c38bb26dd70\") "
Sep 10 15:56:53.065535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.065415    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e4d998-cc95-4e59-9192-07bc8417c31c-openshift-service-ca\") pod \"44e4d998-cc95-4e59-9192-07bc8417c31c\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") "
Sep 10 15:56:53.065535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.065446    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-metrics-certs\") pod \"44e4d998-cc95-4e59-9192-07bc8417c31c\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") "
Sep 10 15:56:53.065535 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.065474    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b92nf\" (UniqueName: \"kubernetes.io/projected/44e4d998-cc95-4e59-9192-07bc8417c31c-kube-api-access-b92nf\") pod \"44e4d998-cc95-4e59-9192-07bc8417c31c\" (UID: \"44e4d998-cc95-4e59-9192-07bc8417c31c\") "
Sep 10 15:56:53.065853 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.065823    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44e4d998-cc95-4e59-9192-07bc8417c31c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "44e4d998-cc95-4e59-9192-07bc8417c31c" (UID: "44e4d998-cc95-4e59-9192-07bc8417c31c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:56:53.065915 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.065856    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e5665f4-3d83-407a-b63d-7c38bb26dd70-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e5665f4-3d83-407a-b63d-7c38bb26dd70" (UID: "8e5665f4-3d83-407a-b63d-7c38bb26dd70"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:56:53.067803 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.067762    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e4d998-cc95-4e59-9192-07bc8417c31c-kube-api-access-b92nf" (OuterVolumeSpecName: "kube-api-access-b92nf") pod "44e4d998-cc95-4e59-9192-07bc8417c31c" (UID: "44e4d998-cc95-4e59-9192-07bc8417c31c"). InnerVolumeSpecName "kube-api-access-b92nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:56:53.067803 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.067794    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e5665f4-3d83-407a-b63d-7c38bb26dd70-kube-api-access-hjvw6" (OuterVolumeSpecName: "kube-api-access-hjvw6") pod "8e5665f4-3d83-407a-b63d-7c38bb26dd70" (UID: "8e5665f4-3d83-407a-b63d-7c38bb26dd70"). InnerVolumeSpecName "kube-api-access-hjvw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:56:53.067978 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.067811    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "44e4d998-cc95-4e59-9192-07bc8417c31c" (UID: "44e4d998-cc95-4e59-9192-07bc8417c31c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:56:53.067978 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.067832    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "44e4d998-cc95-4e59-9192-07bc8417c31c" (UID: "44e4d998-cc95-4e59-9192-07bc8417c31c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:56:53.166971 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.166878    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:53.166971 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.166910    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e5665f4-3d83-407a-b63d-7c38bb26dd70-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:53.166971 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.166920    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjvw6\" (UniqueName: \"kubernetes.io/projected/8e5665f4-3d83-407a-b63d-7c38bb26dd70-kube-api-access-hjvw6\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:53.166971 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.166930    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e4d998-cc95-4e59-9192-07bc8417c31c-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:53.166971 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.166940    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4d998-cc95-4e59-9192-07bc8417c31c-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:53.166971 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.166948    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b92nf\" (UniqueName: \"kubernetes.io/projected/44e4d998-cc95-4e59-9192-07bc8417c31c-kube-api-access-b92nf\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:56:53.401798 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.401762    2527 generic.go:356] "Generic (PLEG): container finished" podID="44e4d998-cc95-4e59-9192-07bc8417c31c" containerID="7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea" exitCode=0
Sep 10 15:56:53.402007 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.401826    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"
Sep 10 15:56:53.402007 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.401856    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2" event={"ID":"44e4d998-cc95-4e59-9192-07bc8417c31c","Type":"ContainerDied","Data":"7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea"}
Sep 10 15:56:53.402007 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.401947    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2" event={"ID":"44e4d998-cc95-4e59-9192-07bc8417c31c","Type":"ContainerDied","Data":"0c97946113281847d88145fc96403d8e421d06b98fbd7969978490cf19095e81"}
Sep 10 15:56:53.402007 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.401975    2527 scope.go:117] "RemoveContainer" containerID="7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea"
Sep 10 15:56:53.403327 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.403299    2527 generic.go:356] "Generic (PLEG): container finished" podID="8e5665f4-3d83-407a-b63d-7c38bb26dd70" containerID="3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f" exitCode=0
Sep 10 15:56:53.403429 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.403371    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"
Sep 10 15:56:53.403429 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.403385    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v" event={"ID":"8e5665f4-3d83-407a-b63d-7c38bb26dd70","Type":"ContainerDied","Data":"3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f"}
Sep 10 15:56:53.403562 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.403444    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v" event={"ID":"8e5665f4-3d83-407a-b63d-7c38bb26dd70","Type":"ContainerDied","Data":"277e313ae154903a211ea49c167fee1bd7060a1dd561fc0fb28c2b3e6ac2e272"}
Sep 10 15:56:53.413158 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.413110    2527 scope.go:117] "RemoveContainer" containerID="7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea"
Sep 10 15:56:53.413503 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:56:53.413478    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea\": container with ID starting with 7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea not found: ID does not exist" containerID="7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea"
Sep 10 15:56:53.413585 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.413519    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea"} err="failed to get container status \"7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea\": rpc error: code = NotFound desc = could not find container \"7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea\": container with ID starting with 7a4ae42388713c1b3fa222223435eb3b45c2b5fead3766f5db5f634f65a7d8ea not found: ID does not exist"
Sep 10 15:56:53.413585 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.413544    2527 scope.go:117] "RemoveContainer" containerID="3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f"
Sep 10 15:56:53.422423 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.422400    2527 scope.go:117] "RemoveContainer" containerID="3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f"
Sep 10 15:56:53.422719 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:56:53.422699    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f\": container with ID starting with 3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f not found: ID does not exist" containerID="3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f"
Sep 10 15:56:53.422760 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.422728    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f"} err="failed to get container status \"3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f\": rpc error: code = NotFound desc = could not find container \"3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f\": container with ID starting with 3ff2338f4e46bf60e3fa4edfaaa698a04f2eff5f4cb10ff0b29ee4d027d5940f not found: ID does not exist"
Sep 10 15:56:53.429313 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.429286    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"]
Sep 10 15:56:53.433666 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.433640    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2"]
Sep 10 15:56:53.444860 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.444828    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"]
Sep 10 15:56:53.451944 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:53.451901    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-314824-f459986f8-bfz8v"]
Sep 10 15:56:54.378200 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:54.378170    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:56:54.566011 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:54.565972    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44e4d998-cc95-4e59-9192-07bc8417c31c" path="/var/lib/kubelet/pods/44e4d998-cc95-4e59-9192-07bc8417c31c/volumes"
Sep 10 15:56:54.566463 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:54.566445    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e5665f4-3d83-407a-b63d-7c38bb26dd70" path="/var/lib/kubelet/pods/8e5665f4-3d83-407a-b63d-7c38bb26dd70/volumes"
Sep 10 15:56:55.864677 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:56:55.864621    2527 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-autores-314824-7588dc75b9-kx7h2" podUID="44e4d998-cc95-4e59-9192-07bc8417c31c" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.46:8443/health\": context deadline exceeded"
Sep 10 15:57:00.380835 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:00.380802    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:57:17.660008 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:17.659969    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"]
Sep 10 15:57:17.660686 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:17.660260    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9" podUID="72bc2ceb-6487-4314-8323-d5aad8436dd0" containerName="maas-api" containerID="cri-o://859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80" gracePeriod=30
Sep 10 15:57:17.718861 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:17.718819    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"]
Sep 10 15:57:17.719110 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:17.719074    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg" podUID="a5cba170-19bf-4cbe-adea-8eb7ecf84466" containerName="payload-processing" containerID="cri-o://dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c" gracePeriod=30
Sep 10 15:57:17.977544 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:17.977515    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:57:18.015464 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.015431    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:57:18.116619 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.116586    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfkr8\" (UniqueName: \"kubernetes.io/projected/72bc2ceb-6487-4314-8323-d5aad8436dd0-kube-api-access-hfkr8\") pod \"72bc2ceb-6487-4314-8323-d5aad8436dd0\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") "
Sep 10 15:57:18.116806 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.116645    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bc2ceb-6487-4314-8323-d5aad8436dd0-openshift-service-ca\") pod \"72bc2ceb-6487-4314-8323-d5aad8436dd0\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") "
Sep 10 15:57:18.116806 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.116666    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-metrics-certs\") pod \"72bc2ceb-6487-4314-8323-d5aad8436dd0\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") "
Sep 10 15:57:18.116806 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.116722    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-maas-api-tls\") pod \"72bc2ceb-6487-4314-8323-d5aad8436dd0\" (UID: \"72bc2ceb-6487-4314-8323-d5aad8436dd0\") "
Sep 10 15:57:18.116806 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.116775    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5mg7\" (UniqueName: \"kubernetes.io/projected/a5cba170-19bf-4cbe-adea-8eb7ecf84466-kube-api-access-z5mg7\") pod \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\" (UID: \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\") "
Sep 10 15:57:18.116806 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.116792    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5cba170-19bf-4cbe-adea-8eb7ecf84466-plugins-config-volume\") pod \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\" (UID: \"a5cba170-19bf-4cbe-adea-8eb7ecf84466\") "
Sep 10 15:57:18.117197 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.117018    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72bc2ceb-6487-4314-8323-d5aad8436dd0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "72bc2ceb-6487-4314-8323-d5aad8436dd0" (UID: "72bc2ceb-6487-4314-8323-d5aad8436dd0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:57:18.117450 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.117422    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5cba170-19bf-4cbe-adea-8eb7ecf84466-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a5cba170-19bf-4cbe-adea-8eb7ecf84466" (UID: "a5cba170-19bf-4cbe-adea-8eb7ecf84466"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:57:18.119352 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.119322    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "72bc2ceb-6487-4314-8323-d5aad8436dd0" (UID: "72bc2ceb-6487-4314-8323-d5aad8436dd0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:57:18.119352 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.119350    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72bc2ceb-6487-4314-8323-d5aad8436dd0-kube-api-access-hfkr8" (OuterVolumeSpecName: "kube-api-access-hfkr8") pod "72bc2ceb-6487-4314-8323-d5aad8436dd0" (UID: "72bc2ceb-6487-4314-8323-d5aad8436dd0"). InnerVolumeSpecName "kube-api-access-hfkr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:57:18.119573 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.119558    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "72bc2ceb-6487-4314-8323-d5aad8436dd0" (UID: "72bc2ceb-6487-4314-8323-d5aad8436dd0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:57:18.119854 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.119839    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5cba170-19bf-4cbe-adea-8eb7ecf84466-kube-api-access-z5mg7" (OuterVolumeSpecName: "kube-api-access-z5mg7") pod "a5cba170-19bf-4cbe-adea-8eb7ecf84466" (UID: "a5cba170-19bf-4cbe-adea-8eb7ecf84466"). InnerVolumeSpecName "kube-api-access-z5mg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:57:18.217734 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.217640    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfkr8\" (UniqueName: \"kubernetes.io/projected/72bc2ceb-6487-4314-8323-d5aad8436dd0-kube-api-access-hfkr8\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:18.217734 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.217677    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72bc2ceb-6487-4314-8323-d5aad8436dd0-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:18.217734 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.217687    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:18.217734 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.217697    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72bc2ceb-6487-4314-8323-d5aad8436dd0-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:18.217734 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.217705    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5mg7\" (UniqueName: \"kubernetes.io/projected/a5cba170-19bf-4cbe-adea-8eb7ecf84466-kube-api-access-z5mg7\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:18.217734 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.217715    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5cba170-19bf-4cbe-adea-8eb7ecf84466-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:18.546246 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.546199    2527 generic.go:356] "Generic (PLEG): container finished" podID="a5cba170-19bf-4cbe-adea-8eb7ecf84466" containerID="dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c" exitCode=0
Sep 10 15:57:18.546246 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.546252    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"
Sep 10 15:57:18.546538 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.546294    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg" event={"ID":"a5cba170-19bf-4cbe-adea-8eb7ecf84466","Type":"ContainerDied","Data":"dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c"}
Sep 10 15:57:18.546538 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.546357    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg" event={"ID":"a5cba170-19bf-4cbe-adea-8eb7ecf84466","Type":"ContainerDied","Data":"b65753013358b6a51b25d4e4af943e0af033bc997c3adad47c202d57f53a4651"}
Sep 10 15:57:18.546538 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.546380    2527 scope.go:117] "RemoveContainer" containerID="dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c"
Sep 10 15:57:18.547976 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.547943    2527 generic.go:356] "Generic (PLEG): container finished" podID="72bc2ceb-6487-4314-8323-d5aad8436dd0" containerID="859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80" exitCode=0
Sep 10 15:57:18.548081 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.547995    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"
Sep 10 15:57:18.548081 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.548068    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9" event={"ID":"72bc2ceb-6487-4314-8323-d5aad8436dd0","Type":"ContainerDied","Data":"859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80"}
Sep 10 15:57:18.548204 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.548114    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9" event={"ID":"72bc2ceb-6487-4314-8323-d5aad8436dd0","Type":"ContainerDied","Data":"235dadbb84eb4619b36478795f06bce1c79b436ae507423e60864719f561a15f"}
Sep 10 15:57:18.557171 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.557113    2527 scope.go:117] "RemoveContainer" containerID="dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c"
Sep 10 15:57:18.557541 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:57:18.557517    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c\": container with ID starting with dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c not found: ID does not exist" containerID="dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c"
Sep 10 15:57:18.557606 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.557558    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c"} err="failed to get container status \"dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c\": rpc error: code = NotFound desc = could not find container \"dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c\": container with ID starting with dd5d14eb8808e1c34c7d39e59c48f6f5ff16b97d4f13bcbceb735fa82a02097c not found: ID does not exist"
Sep 10 15:57:18.557606 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.557585    2527 scope.go:117] "RemoveContainer" containerID="859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80"
Sep 10 15:57:18.575439 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.575407    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"]
Sep 10 15:57:18.576802 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.576772    2527 scope.go:117] "RemoveContainer" containerID="859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80"
Sep 10 15:57:18.577203 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:57:18.577179    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80\": container with ID starting with 859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80 not found: ID does not exist" containerID="859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80"
Sep 10 15:57:18.577315 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.577211    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80"} err="failed to get container status \"859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80\": rpc error: code = NotFound desc = could not find container \"859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80\": container with ID starting with 859accf8d82e81f43c54cd08a16208628b3725a93d33d47ebcb8e0e17f95ed80 not found: ID does not exist"
Sep 10 15:57:18.581298 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.581272    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-529c3efe-8465c94579-hzdkg"]
Sep 10 15:57:18.594077 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.594026    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"]
Sep 10 15:57:18.596106 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:18.596077    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-529c3efe-59dffc7f6-zllk9"]
Sep 10 15:57:20.564657 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:20.564614    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72bc2ceb-6487-4314-8323-d5aad8436dd0" path="/var/lib/kubelet/pods/72bc2ceb-6487-4314-8323-d5aad8436dd0/volumes"
Sep 10 15:57:20.567228 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:20.565044    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5cba170-19bf-4cbe-adea-8eb7ecf84466" path="/var/lib/kubelet/pods/a5cba170-19bf-4cbe-adea-8eb7ecf84466/volumes"
Sep 10 15:57:23.138574 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.138535    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"]
Sep 10 15:57:23.139032 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139016    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5665f4-3d83-407a-b63d-7c38bb26dd70" containerName="payload-pre-processing"
Sep 10 15:57:23.139076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139034    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5665f4-3d83-407a-b63d-7c38bb26dd70" containerName="payload-pre-processing"
Sep 10 15:57:23.139076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139046    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72bc2ceb-6487-4314-8323-d5aad8436dd0" containerName="maas-api"
Sep 10 15:57:23.139076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139052    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="72bc2ceb-6487-4314-8323-d5aad8436dd0" containerName="maas-api"
Sep 10 15:57:23.139076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139069    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e4d998-cc95-4e59-9192-07bc8417c31c" containerName="maas-api"
Sep 10 15:57:23.139076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139076    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e4d998-cc95-4e59-9192-07bc8417c31c" containerName="maas-api"
Sep 10 15:57:23.139259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139082    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5cba170-19bf-4cbe-adea-8eb7ecf84466" containerName="payload-processing"
Sep 10 15:57:23.139259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139089    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5cba170-19bf-4cbe-adea-8eb7ecf84466" containerName="payload-processing"
Sep 10 15:57:23.139259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139180    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5665f4-3d83-407a-b63d-7c38bb26dd70" containerName="payload-pre-processing"
Sep 10 15:57:23.139259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139192    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5cba170-19bf-4cbe-adea-8eb7ecf84466" containerName="payload-processing"
Sep 10 15:57:23.139259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139199    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e4d998-cc95-4e59-9192-07bc8417c31c" containerName="maas-api"
Sep 10 15:57:23.139259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.139204    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="72bc2ceb-6487-4314-8323-d5aad8436dd0" containerName="maas-api"
Sep 10 15:57:23.142708 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.142684    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.145395 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.145067    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7a3187a1-serving-cert\""
Sep 10 15:57:23.153674 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.153645    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"]
Sep 10 15:57:23.267769 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.267725    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l8rf\" (UniqueName: \"kubernetes.io/projected/efc67ba9-c7dd-4653-8279-0021aa24d6a6-kube-api-access-4l8rf\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.267968 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.267844    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-maas-api-tls\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.267968 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.267905    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efc67ba9-c7dd-4653-8279-0021aa24d6a6-openshift-service-ca\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.267968 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.267929    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-metrics-certs\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.368810 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.368756    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l8rf\" (UniqueName: \"kubernetes.io/projected/efc67ba9-c7dd-4653-8279-0021aa24d6a6-kube-api-access-4l8rf\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.369021 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.368875    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-maas-api-tls\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.369021 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.368945    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efc67ba9-c7dd-4653-8279-0021aa24d6a6-openshift-service-ca\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.369021 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.368969    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-metrics-certs\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.369818 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.369792    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efc67ba9-c7dd-4653-8279-0021aa24d6a6-openshift-service-ca\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.371691 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.371665    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-maas-api-tls\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.371796 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.371781    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-metrics-certs\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.377999 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.377973    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l8rf\" (UniqueName: \"kubernetes.io/projected/efc67ba9-c7dd-4653-8279-0021aa24d6a6-kube-api-access-4l8rf\") pod \"maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.459541 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.459443    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:23.743686 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.743625    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"]
Sep 10 15:57:23.746213 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:57:23.746182    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc67ba9_c7dd_4653_8279_0021aa24d6a6.slice/crio-33d524e007ac05671cb5c8ef90f174549163f3133005f3b404f89a25aacaba8b WatchSource:0}: Error finding container 33d524e007ac05671cb5c8ef90f174549163f3133005f3b404f89a25aacaba8b: Status 404 returned error can't find the container with id 33d524e007ac05671cb5c8ef90f174549163f3133005f3b404f89a25aacaba8b
Sep 10 15:57:23.973291 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.973253    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"]
Sep 10 15:57:23.978306 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.978280    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:23.980859 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.980834    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-7a3187a1\""
Sep 10 15:57:23.989898 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:23.989869    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"]
Sep 10 15:57:24.075092 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.075054    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4169e93f-ce1c-4043-92fd-e8a9deddddc4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs\" (UID: \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:24.075344 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.075113    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlcm7\" (UniqueName: \"kubernetes.io/projected/4169e93f-ce1c-4043-92fd-e8a9deddddc4-kube-api-access-mlcm7\") pod \"payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs\" (UID: \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:24.176580 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.176546    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4169e93f-ce1c-4043-92fd-e8a9deddddc4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs\" (UID: \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:24.176914 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.176639    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlcm7\" (UniqueName: \"kubernetes.io/projected/4169e93f-ce1c-4043-92fd-e8a9deddddc4-kube-api-access-mlcm7\") pod \"payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs\" (UID: \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:24.177572 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.177549    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4169e93f-ce1c-4043-92fd-e8a9deddddc4-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs\" (UID: \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:24.186317 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.186295    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlcm7\" (UniqueName: \"kubernetes.io/projected/4169e93f-ce1c-4043-92fd-e8a9deddddc4-kube-api-access-mlcm7\") pod \"payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs\" (UID: \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:24.291314 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.291284    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:24.569063 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.569036    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"]
Sep 10 15:57:24.571113 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:57:24.571082    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4169e93f_ce1c_4043_92fd_e8a9deddddc4.slice/crio-d6811b580dd87bc48fb812a1475b44287cc020c07d5f148a6fe31ffae38b483d WatchSource:0}: Error finding container d6811b580dd87bc48fb812a1475b44287cc020c07d5f148a6fe31ffae38b483d: Status 404 returned error can't find the container with id d6811b580dd87bc48fb812a1475b44287cc020c07d5f148a6fe31ffae38b483d
Sep 10 15:57:24.589201 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.589125    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk" event={"ID":"efc67ba9-c7dd-4653-8279-0021aa24d6a6","Type":"ContainerStarted","Data":"6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb"}
Sep 10 15:57:24.589359 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.589269    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk" event={"ID":"efc67ba9-c7dd-4653-8279-0021aa24d6a6","Type":"ContainerStarted","Data":"33d524e007ac05671cb5c8ef90f174549163f3133005f3b404f89a25aacaba8b"}
Sep 10 15:57:24.589359 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.589302    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:24.590530 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.590504    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs" event={"ID":"4169e93f-ce1c-4043-92fd-e8a9deddddc4","Type":"ContainerStarted","Data":"d6811b580dd87bc48fb812a1475b44287cc020c07d5f148a6fe31ffae38b483d"}
Sep 10 15:57:24.612288 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:24.612202    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk" podStartSLOduration=1.187986926 podStartE2EDuration="1.61217741s" totalImagesPullingTime="424.190484ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:24.610423083 +0000 UTC m=+940.526547963" watchObservedRunningTime="2026-09-10 15:57:24.61217741 +0000 UTC m=+940.528302317"
Sep 10 15:57:25.597976 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:25.597938    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs" event={"ID":"4169e93f-ce1c-4043-92fd-e8a9deddddc4","Type":"ContainerStarted","Data":"4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24"}
Sep 10 15:57:25.616687 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:25.616626    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs" podStartSLOduration=2.616604982 podStartE2EDuration="2.616604982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:25.615458525 +0000 UTC m=+941.531583418" watchObservedRunningTime="2026-09-10 15:57:25.616604982 +0000 UTC m=+941.532729860"
Sep 10 15:57:26.603899 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:26.603865    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:30.604599 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:30.604568    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:37.611552 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:37.611515    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:40.830316 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:40.830261    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"]
Sep 10 15:57:40.834242 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:40.834213    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:40.836802 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:40.836764    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-5c1d26-serving-cert\""
Sep 10 15:57:40.843873 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:40.843841    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"]
Sep 10 15:57:40.923754 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:40.923709    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-metrics-certs\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:40.923952 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:40.923768    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-maas-api-tls\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:40.923952 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:40.923849    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjqf6\" (UniqueName: \"kubernetes.io/projected/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-kube-api-access-gjqf6\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:40.924029 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:40.923974    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-openshift-service-ca\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.025237 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.025193    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-openshift-service-ca\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.025439 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.025252    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-metrics-certs\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.025439 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.025284    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-maas-api-tls\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.025439 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.025308    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjqf6\" (UniqueName: \"kubernetes.io/projected/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-kube-api-access-gjqf6\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.026071 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.026031    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-openshift-service-ca\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.028032 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.028002    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-metrics-certs\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.028177 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.028064    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-maas-api-tls\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.033569 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.033544    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjqf6\" (UniqueName: \"kubernetes.io/projected/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-kube-api-access-gjqf6\") pod \"maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.146198 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.146057    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:41.432229 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.432203    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"]
Sep 10 15:57:41.434869 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:57:41.434839    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a90c44c_a616_4cf6_a9b7_6f3b679408c7.slice/crio-6bb6ff35668b15f05d361fdaa12f029786885968e7df993050c76757319d02f2 WatchSource:0}: Error finding container 6bb6ff35668b15f05d361fdaa12f029786885968e7df993050c76757319d02f2: Status 404 returned error can't find the container with id 6bb6ff35668b15f05d361fdaa12f029786885968e7df993050c76757319d02f2
Sep 10 15:57:41.689325 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.689232    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc" event={"ID":"5a90c44c-a616-4cf6-a9b7-6f3b679408c7","Type":"ContainerStarted","Data":"6bb6ff35668b15f05d361fdaa12f029786885968e7df993050c76757319d02f2"}
Sep 10 15:57:41.765893 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.765851    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"]
Sep 10 15:57:41.770737 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.770715    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:41.778914 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.778866    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-5c1d26-dockercfg-hvws8\""
Sep 10 15:57:41.779088 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.778925    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-5c1d26\""
Sep 10 15:57:41.782743 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.782713    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"]
Sep 10 15:57:41.833215 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.833167    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb8jm\" (UniqueName: \"kubernetes.io/projected/ceef1d38-ab9e-4963-b064-76e1e38f2d82-kube-api-access-gb8jm\") pod \"payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw\" (UID: \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:41.833215 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.833214    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceef1d38-ab9e-4963-b064-76e1e38f2d82-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw\" (UID: \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:41.934668 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.934628    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb8jm\" (UniqueName: \"kubernetes.io/projected/ceef1d38-ab9e-4963-b064-76e1e38f2d82-kube-api-access-gb8jm\") pod \"payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw\" (UID: \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:41.934802 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.934688    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceef1d38-ab9e-4963-b064-76e1e38f2d82-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw\" (UID: \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:41.935455 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.935437    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceef1d38-ab9e-4963-b064-76e1e38f2d82-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw\" (UID: \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:41.942876 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:41.942804    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb8jm\" (UniqueName: \"kubernetes.io/projected/ceef1d38-ab9e-4963-b064-76e1e38f2d82-kube-api-access-gb8jm\") pod \"payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw\" (UID: \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:42.089987 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:42.089947    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:42.370468 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:42.370438    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"]
Sep 10 15:57:42.372427 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:57:42.372393    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceef1d38_ab9e_4963_b064_76e1e38f2d82.slice/crio-464c5f1981d73595540e6d67a4e201781a6adf9033f64c0bff38f606e46e3099 WatchSource:0}: Error finding container 464c5f1981d73595540e6d67a4e201781a6adf9033f64c0bff38f606e46e3099: Status 404 returned error can't find the container with id 464c5f1981d73595540e6d67a4e201781a6adf9033f64c0bff38f606e46e3099
Sep 10 15:57:42.697148 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:42.697029    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw" event={"ID":"ceef1d38-ab9e-4963-b064-76e1e38f2d82","Type":"ContainerStarted","Data":"f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f"}
Sep 10 15:57:42.697148 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:42.697095    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw" event={"ID":"ceef1d38-ab9e-4963-b064-76e1e38f2d82","Type":"ContainerStarted","Data":"464c5f1981d73595540e6d67a4e201781a6adf9033f64c0bff38f606e46e3099"}
Sep 10 15:57:42.697392 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:42.697128    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:42.698762 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:42.698733    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc" event={"ID":"5a90c44c-a616-4cf6-a9b7-6f3b679408c7","Type":"ContainerStarted","Data":"7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13"}
Sep 10 15:57:42.699163 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:42.698821    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:42.712468 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:42.712407    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw" podStartSLOduration=1.712389095 podStartE2EDuration="1.712389095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:42.711515955 +0000 UTC m=+958.627640849" watchObservedRunningTime="2026-09-10 15:57:42.712389095 +0000 UTC m=+958.628513965"
Sep 10 15:57:42.729868 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:42.729797    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc" podStartSLOduration=2.29560439 podStartE2EDuration="2.729775363s" totalImagesPullingTime="434.170973ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:57:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:57:42.727386138 +0000 UTC m=+958.643511017" watchObservedRunningTime="2026-09-10 15:57:42.729775363 +0000 UTC m=+958.645900242"
Sep 10 15:57:48.711259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:48.711225    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:57:53.703392 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:53.703348    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"]
Sep 10 15:57:53.703853 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:53.703699    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk" podUID="efc67ba9-c7dd-4653-8279-0021aa24d6a6" containerName="maas-api" containerID="cri-o://6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb" gracePeriod=30
Sep 10 15:57:53.709366 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:53.708874    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:57:53.784310 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:53.784269    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"]
Sep 10 15:57:53.784532 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:53.784508    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs" podUID="4169e93f-ce1c-4043-92fd-e8a9deddddc4" containerName="payload-pre-processing" containerID="cri-o://4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24" gracePeriod=30
Sep 10 15:57:54.055377 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.055349    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:54.092418 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.092385    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:54.152291 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.152207    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4169e93f-ce1c-4043-92fd-e8a9deddddc4-plugins-config-volume\") pod \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\" (UID: \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\") "
Sep 10 15:57:54.152291 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.152270    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-maas-api-tls\") pod \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") "
Sep 10 15:57:54.152598 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.152353    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-metrics-certs\") pod \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") "
Sep 10 15:57:54.152598 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.152387    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efc67ba9-c7dd-4653-8279-0021aa24d6a6-openshift-service-ca\") pod \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") "
Sep 10 15:57:54.152598 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.152420    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlcm7\" (UniqueName: \"kubernetes.io/projected/4169e93f-ce1c-4043-92fd-e8a9deddddc4-kube-api-access-mlcm7\") pod \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\" (UID: \"4169e93f-ce1c-4043-92fd-e8a9deddddc4\") "
Sep 10 15:57:54.152598 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.152486    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l8rf\" (UniqueName: \"kubernetes.io/projected/efc67ba9-c7dd-4653-8279-0021aa24d6a6-kube-api-access-4l8rf\") pod \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\" (UID: \"efc67ba9-c7dd-4653-8279-0021aa24d6a6\") "
Sep 10 15:57:54.153388 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.153095    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efc67ba9-c7dd-4653-8279-0021aa24d6a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efc67ba9-c7dd-4653-8279-0021aa24d6a6" (UID: "efc67ba9-c7dd-4653-8279-0021aa24d6a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:57:54.153388 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.153349    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4169e93f-ce1c-4043-92fd-e8a9deddddc4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4169e93f-ce1c-4043-92fd-e8a9deddddc4" (UID: "4169e93f-ce1c-4043-92fd-e8a9deddddc4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:57:54.155858 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.155823    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4169e93f-ce1c-4043-92fd-e8a9deddddc4-kube-api-access-mlcm7" (OuterVolumeSpecName: "kube-api-access-mlcm7") pod "4169e93f-ce1c-4043-92fd-e8a9deddddc4" (UID: "4169e93f-ce1c-4043-92fd-e8a9deddddc4"). InnerVolumeSpecName "kube-api-access-mlcm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:57:54.157023 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.156992    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efc67ba9-c7dd-4653-8279-0021aa24d6a6-kube-api-access-4l8rf" (OuterVolumeSpecName: "kube-api-access-4l8rf") pod "efc67ba9-c7dd-4653-8279-0021aa24d6a6" (UID: "efc67ba9-c7dd-4653-8279-0021aa24d6a6"). InnerVolumeSpecName "kube-api-access-4l8rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:57:54.158333 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.158305    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efc67ba9-c7dd-4653-8279-0021aa24d6a6" (UID: "efc67ba9-c7dd-4653-8279-0021aa24d6a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:57:54.158483 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.158443    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "efc67ba9-c7dd-4653-8279-0021aa24d6a6" (UID: "efc67ba9-c7dd-4653-8279-0021aa24d6a6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:57:54.253863 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.253811    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4169e93f-ce1c-4043-92fd-e8a9deddddc4-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:54.253863 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.253854    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:54.253863 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.253867    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efc67ba9-c7dd-4653-8279-0021aa24d6a6-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:54.254125 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.253877    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efc67ba9-c7dd-4653-8279-0021aa24d6a6-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:54.254125 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.253890    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlcm7\" (UniqueName: \"kubernetes.io/projected/4169e93f-ce1c-4043-92fd-e8a9deddddc4-kube-api-access-mlcm7\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:54.254125 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.253902    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l8rf\" (UniqueName: \"kubernetes.io/projected/efc67ba9-c7dd-4653-8279-0021aa24d6a6-kube-api-access-4l8rf\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:57:54.774378 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.774339    2527 generic.go:356] "Generic (PLEG): container finished" podID="4169e93f-ce1c-4043-92fd-e8a9deddddc4" containerID="4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24" exitCode=0
Sep 10 15:57:54.774850 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.774393    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"
Sep 10 15:57:54.774850 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.774440    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs" event={"ID":"4169e93f-ce1c-4043-92fd-e8a9deddddc4","Type":"ContainerDied","Data":"4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24"}
Sep 10 15:57:54.774850 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.774503    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs" event={"ID":"4169e93f-ce1c-4043-92fd-e8a9deddddc4","Type":"ContainerDied","Data":"d6811b580dd87bc48fb812a1475b44287cc020c07d5f148a6fe31ffae38b483d"}
Sep 10 15:57:54.774850 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.774522    2527 scope.go:117] "RemoveContainer" containerID="4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24"
Sep 10 15:57:54.776319 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.776296    2527 generic.go:356] "Generic (PLEG): container finished" podID="efc67ba9-c7dd-4653-8279-0021aa24d6a6" containerID="6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb" exitCode=0
Sep 10 15:57:54.776451 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.776337    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"
Sep 10 15:57:54.776451 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.776344    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk" event={"ID":"efc67ba9-c7dd-4653-8279-0021aa24d6a6","Type":"ContainerDied","Data":"6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb"}
Sep 10 15:57:54.776451 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.776392    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk" event={"ID":"efc67ba9-c7dd-4653-8279-0021aa24d6a6","Type":"ContainerDied","Data":"33d524e007ac05671cb5c8ef90f174549163f3133005f3b404f89a25aacaba8b"}
Sep 10 15:57:54.784750 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.784726    2527 scope.go:117] "RemoveContainer" containerID="4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24"
Sep 10 15:57:54.785106 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:57:54.785082    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24\": container with ID starting with 4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24 not found: ID does not exist" containerID="4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24"
Sep 10 15:57:54.785285 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.785115    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24"} err="failed to get container status \"4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24\": rpc error: code = NotFound desc = could not find container \"4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24\": container with ID starting with 4518977ea0e98542e742950f5def635b8a4a4622bfecddd23f3a8d16b96d4e24 not found: ID does not exist"
Sep 10 15:57:54.785285 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.785176    2527 scope.go:117] "RemoveContainer" containerID="6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb"
Sep 10 15:57:54.796757 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.796735    2527 scope.go:117] "RemoveContainer" containerID="6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb"
Sep 10 15:57:54.797178 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:57:54.797156    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb\": container with ID starting with 6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb not found: ID does not exist" containerID="6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb"
Sep 10 15:57:54.797245 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.797187    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb"} err="failed to get container status \"6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb\": rpc error: code = NotFound desc = could not find container \"6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb\": container with ID starting with 6db0af1a136da0b8ab2ec41cad102d37b97f9c7ace3cb2977f8c9f7dbf62e1eb not found: ID does not exist"
Sep 10 15:57:54.804962 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.804931    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"]
Sep 10 15:57:54.810450 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.810419    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-7a3187a1-7d69cc7b4b-fbkzs"]
Sep 10 15:57:54.822079 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.822050    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"]
Sep 10 15:57:54.826350 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:54.826274    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a3187a1-846d4cfcd8-nqvgk"]
Sep 10 15:57:56.564182 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:56.564124    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4169e93f-ce1c-4043-92fd-e8a9deddddc4" path="/var/lib/kubelet/pods/4169e93f-ce1c-4043-92fd-e8a9deddddc4/volumes"
Sep 10 15:57:56.564580 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:57:56.564567    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efc67ba9-c7dd-4653-8279-0021aa24d6a6" path="/var/lib/kubelet/pods/efc67ba9-c7dd-4653-8279-0021aa24d6a6/volumes"
Sep 10 15:58:25.248722 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.248685    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-745779c994-vf2p6"]
Sep 10 15:58:25.249377 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.249349    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efc67ba9-c7dd-4653-8279-0021aa24d6a6" containerName="maas-api"
Sep 10 15:58:25.249377 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.249374    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="efc67ba9-c7dd-4653-8279-0021aa24d6a6" containerName="maas-api"
Sep 10 15:58:25.249511 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.249390    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4169e93f-ce1c-4043-92fd-e8a9deddddc4" containerName="payload-pre-processing"
Sep 10 15:58:25.249511 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.249400    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="4169e93f-ce1c-4043-92fd-e8a9deddddc4" containerName="payload-pre-processing"
Sep 10 15:58:25.249511 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.249502    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="efc67ba9-c7dd-4653-8279-0021aa24d6a6" containerName="maas-api"
Sep 10 15:58:25.249635 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.249519    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="4169e93f-ce1c-4043-92fd-e8a9deddddc4" containerName="payload-pre-processing"
Sep 10 15:58:25.253089 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.253064    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.256638 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.256614    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 15:58:25.256826 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.256614    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 15:58:25.256903 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.256645    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mkfbn\""
Sep 10 15:58:25.256903 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.256667    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 15:58:25.257007 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.256691    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 15:58:25.257575 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.257559    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 15:58:25.267505 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.267473    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-745779c994-vf2p6"]
Sep 10 15:58:25.270564 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.270541    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 15:58:25.354583 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.354543    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-service-ca\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.354776 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.354598    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dgl4\" (UniqueName: \"kubernetes.io/projected/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-kube-api-access-6dgl4\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.354776 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.354676    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-console-serving-cert\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.354776 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.354710    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-oauth-serving-cert\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.354776 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.354754    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-trusted-ca-bundle\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.354917 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.354781    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-console-config\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.354917 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.354818    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-console-oauth-config\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.455556 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.455518    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-console-serving-cert\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.455556 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.455558    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-oauth-serving-cert\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.455788 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.455662    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-trusted-ca-bundle\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.455788 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.455690    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-console-config\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.455788 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.455720    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-console-oauth-config\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.455944 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.455791    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-service-ca\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.455944 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.455893    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dgl4\" (UniqueName: \"kubernetes.io/projected/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-kube-api-access-6dgl4\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.456515 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.456488    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-oauth-serving-cert\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.456651 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.456566    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-console-config\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.456746 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.456722    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-trusted-ca-bundle\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.456814 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.456722    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-service-ca\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.458399 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.458374    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-console-oauth-config\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.458531 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.458511    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-console-serving-cert\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.464939 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.464909    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dgl4\" (UniqueName: \"kubernetes.io/projected/3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79-kube-api-access-6dgl4\") pod \"console-745779c994-vf2p6\" (UID: \"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79\") " pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.564672 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.564635    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:25.841990 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.841956    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-745779c994-vf2p6"]
Sep 10 15:58:25.843645 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:58:25.843616    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3f9eb2_cdac_468d_bf2d_c0faf0f03b79.slice/crio-49713634335d69b3d78a0875f3a17dd5894ba5e5f82be0deb07675b404cd9784 WatchSource:0}: Error finding container 49713634335d69b3d78a0875f3a17dd5894ba5e5f82be0deb07675b404cd9784: Status 404 returned error can't find the container with id 49713634335d69b3d78a0875f3a17dd5894ba5e5f82be0deb07675b404cd9784
Sep 10 15:58:25.950794 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.950754    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-745779c994-vf2p6" event={"ID":"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79","Type":"ContainerStarted","Data":"e15df14a6f29487fc433e359fb9f206b489f0a0bce901ba13f752eb6bbd519e4"}
Sep 10 15:58:25.951016 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.950809    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-745779c994-vf2p6" event={"ID":"3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79","Type":"ContainerStarted","Data":"49713634335d69b3d78a0875f3a17dd5894ba5e5f82be0deb07675b404cd9784"}
Sep 10 15:58:25.971126 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:25.971052    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-745779c994-vf2p6" podStartSLOduration=0.971028248 podStartE2EDuration="971.028248ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:58:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:58:25.970287768 +0000 UTC m=+1001.886412684" watchObservedRunningTime="2026-09-10 15:58:25.971028248 +0000 UTC m=+1001.887153128"
Sep 10 15:58:35.565083 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:35.565038    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:35.565555 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:35.565101    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:35.570362 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:35.570333    2527 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:36.019716 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:36.019621    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-745779c994-vf2p6"
Sep 10 15:58:47.213291 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.213250    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"]
Sep 10 15:58:47.213760 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.213531    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc" podUID="5a90c44c-a616-4cf6-a9b7-6f3b679408c7" containerName="maas-api" containerID="cri-o://7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13" gracePeriod=30
Sep 10 15:58:47.305292 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.305249    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"]
Sep 10 15:58:47.305567 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.305536    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw" podUID="ceef1d38-ab9e-4963-b064-76e1e38f2d82" containerName="payload-pre-processing" containerID="cri-o://f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f" gracePeriod=30
Sep 10 15:58:47.590254 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.590222    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:58:47.611470 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.611440    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:58:47.782768 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.782729    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-maas-api-tls\") pod \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") "
Sep 10 15:58:47.782966 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.782772    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-openshift-service-ca\") pod \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") "
Sep 10 15:58:47.782966 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.782858    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceef1d38-ab9e-4963-b064-76e1e38f2d82-plugins-config-volume\") pod \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\" (UID: \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\") "
Sep 10 15:58:47.782966 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.782915    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjqf6\" (UniqueName: \"kubernetes.io/projected/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-kube-api-access-gjqf6\") pod \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") "
Sep 10 15:58:47.782966 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.782940    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb8jm\" (UniqueName: \"kubernetes.io/projected/ceef1d38-ab9e-4963-b064-76e1e38f2d82-kube-api-access-gb8jm\") pod \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\" (UID: \"ceef1d38-ab9e-4963-b064-76e1e38f2d82\") "
Sep 10 15:58:47.783234 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.782980    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-metrics-certs\") pod \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\" (UID: \"5a90c44c-a616-4cf6-a9b7-6f3b679408c7\") "
Sep 10 15:58:47.783297 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.783254    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a90c44c-a616-4cf6-a9b7-6f3b679408c7" (UID: "5a90c44c-a616-4cf6-a9b7-6f3b679408c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:58:47.783360 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.783315    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceef1d38-ab9e-4963-b064-76e1e38f2d82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ceef1d38-ab9e-4963-b064-76e1e38f2d82" (UID: "ceef1d38-ab9e-4963-b064-76e1e38f2d82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:58:47.783417 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.783396    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:58:47.785370 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.785337    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5a90c44c-a616-4cf6-a9b7-6f3b679408c7" (UID: "5a90c44c-a616-4cf6-a9b7-6f3b679408c7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:58:47.785495 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.785350    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a90c44c-a616-4cf6-a9b7-6f3b679408c7" (UID: "5a90c44c-a616-4cf6-a9b7-6f3b679408c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:58:47.785619 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.785591    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceef1d38-ab9e-4963-b064-76e1e38f2d82-kube-api-access-gb8jm" (OuterVolumeSpecName: "kube-api-access-gb8jm") pod "ceef1d38-ab9e-4963-b064-76e1e38f2d82" (UID: "ceef1d38-ab9e-4963-b064-76e1e38f2d82"). InnerVolumeSpecName "kube-api-access-gb8jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:58:47.785660 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.785618    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-kube-api-access-gjqf6" (OuterVolumeSpecName: "kube-api-access-gjqf6") pod "5a90c44c-a616-4cf6-a9b7-6f3b679408c7" (UID: "5a90c44c-a616-4cf6-a9b7-6f3b679408c7"). InnerVolumeSpecName "kube-api-access-gjqf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:58:47.884774 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.884737    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceef1d38-ab9e-4963-b064-76e1e38f2d82-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:58:47.884774 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.884782    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjqf6\" (UniqueName: \"kubernetes.io/projected/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-kube-api-access-gjqf6\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:58:47.884982 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.884793    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb8jm\" (UniqueName: \"kubernetes.io/projected/ceef1d38-ab9e-4963-b064-76e1e38f2d82-kube-api-access-gb8jm\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:58:47.884982 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.884802    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:58:47.884982 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:47.884811    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a90c44c-a616-4cf6-a9b7-6f3b679408c7-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:58:48.079524 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.079431    2527 generic.go:356] "Generic (PLEG): container finished" podID="5a90c44c-a616-4cf6-a9b7-6f3b679408c7" containerID="7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13" exitCode=0
Sep 10 15:58:48.079524 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.079483    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"
Sep 10 15:58:48.079738 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.079522    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc" event={"ID":"5a90c44c-a616-4cf6-a9b7-6f3b679408c7","Type":"ContainerDied","Data":"7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13"}
Sep 10 15:58:48.079738 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.079580    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc" event={"ID":"5a90c44c-a616-4cf6-a9b7-6f3b679408c7","Type":"ContainerDied","Data":"6bb6ff35668b15f05d361fdaa12f029786885968e7df993050c76757319d02f2"}
Sep 10 15:58:48.079738 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.079603    2527 scope.go:117] "RemoveContainer" containerID="7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13"
Sep 10 15:58:48.081085 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.081061    2527 generic.go:356] "Generic (PLEG): container finished" podID="ceef1d38-ab9e-4963-b064-76e1e38f2d82" containerID="f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f" exitCode=0
Sep 10 15:58:48.081187 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.081100    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"
Sep 10 15:58:48.081241 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.081100    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw" event={"ID":"ceef1d38-ab9e-4963-b064-76e1e38f2d82","Type":"ContainerDied","Data":"f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f"}
Sep 10 15:58:48.081279 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.081266    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw" event={"ID":"ceef1d38-ab9e-4963-b064-76e1e38f2d82","Type":"ContainerDied","Data":"464c5f1981d73595540e6d67a4e201781a6adf9033f64c0bff38f606e46e3099"}
Sep 10 15:58:48.089385 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.089217    2527 scope.go:117] "RemoveContainer" containerID="7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13"
Sep 10 15:58:48.089613 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:58:48.089586    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13\": container with ID starting with 7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13 not found: ID does not exist" containerID="7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13"
Sep 10 15:58:48.089694 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.089628    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13"} err="failed to get container status \"7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13\": rpc error: code = NotFound desc = could not find container \"7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13\": container with ID starting with 7a6482058199f1f9aecc8c7aa26a3dedd9dbda3a3c3676b00717a5dbd5fb7a13 not found: ID does not exist"
Sep 10 15:58:48.089694 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.089648    2527 scope.go:117] "RemoveContainer" containerID="f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f"
Sep 10 15:58:48.098324 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.098302    2527 scope.go:117] "RemoveContainer" containerID="f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f"
Sep 10 15:58:48.098636 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:58:48.098619    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f\": container with ID starting with f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f not found: ID does not exist" containerID="f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f"
Sep 10 15:58:48.098679 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.098644    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f"} err="failed to get container status \"f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f\": rpc error: code = NotFound desc = could not find container \"f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f\": container with ID starting with f24ee7c862a4ab016b33d464b582a6727372ccb9feeb27cfe5704ca232d7331f not found: ID does not exist"
Sep 10 15:58:48.109708 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.109674    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"]
Sep 10 15:58:48.115660 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.115628    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5c1d26-7599755b66-fv5tw"]
Sep 10 15:58:48.127390 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.127333    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"]
Sep 10 15:58:48.130083 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.130054    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c1d26-7747577dc7-8hczc"]
Sep 10 15:58:48.571832 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.571794    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a90c44c-a616-4cf6-a9b7-6f3b679408c7" path="/var/lib/kubelet/pods/5a90c44c-a616-4cf6-a9b7-6f3b679408c7/volumes"
Sep 10 15:58:48.572342 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:58:48.572192    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceef1d38-ab9e-4963-b064-76e1e38f2d82" path="/var/lib/kubelet/pods/ceef1d38-ab9e-4963-b064-76e1e38f2d82/volumes"
Sep 10 15:59:29.753259 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:29.753221    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"]
Sep 10 15:59:29.753840 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:29.753721    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb" podUID="453b04b9-0a28-4db4-8fa9-77d296992e3d" containerName="maas-api" containerID="cri-o://d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66" gracePeriod=30
Sep 10 15:59:30.025881 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.025850    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:59:30.182970 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.182926    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/453b04b9-0a28-4db4-8fa9-77d296992e3d-openshift-service-ca\") pod \"453b04b9-0a28-4db4-8fa9-77d296992e3d\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") "
Sep 10 15:59:30.182970 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.182983    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-maas-api-tls\") pod \"453b04b9-0a28-4db4-8fa9-77d296992e3d\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") "
Sep 10 15:59:30.183281 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.183042    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-metrics-certs\") pod \"453b04b9-0a28-4db4-8fa9-77d296992e3d\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") "
Sep 10 15:59:30.183281 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.183085    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5wgh\" (UniqueName: \"kubernetes.io/projected/453b04b9-0a28-4db4-8fa9-77d296992e3d-kube-api-access-p5wgh\") pod \"453b04b9-0a28-4db4-8fa9-77d296992e3d\" (UID: \"453b04b9-0a28-4db4-8fa9-77d296992e3d\") "
Sep 10 15:59:30.183476 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.183447    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/453b04b9-0a28-4db4-8fa9-77d296992e3d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "453b04b9-0a28-4db4-8fa9-77d296992e3d" (UID: "453b04b9-0a28-4db4-8fa9-77d296992e3d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:59:30.185404 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.185370    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "453b04b9-0a28-4db4-8fa9-77d296992e3d" (UID: "453b04b9-0a28-4db4-8fa9-77d296992e3d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:59:30.185404 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.185391    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "453b04b9-0a28-4db4-8fa9-77d296992e3d" (UID: "453b04b9-0a28-4db4-8fa9-77d296992e3d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:59:30.185568 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.185465    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/453b04b9-0a28-4db4-8fa9-77d296992e3d-kube-api-access-p5wgh" (OuterVolumeSpecName: "kube-api-access-p5wgh") pod "453b04b9-0a28-4db4-8fa9-77d296992e3d" (UID: "453b04b9-0a28-4db4-8fa9-77d296992e3d"). InnerVolumeSpecName "kube-api-access-p5wgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:59:30.284603 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.284553    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:30.284603 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.284603    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5wgh\" (UniqueName: \"kubernetes.io/projected/453b04b9-0a28-4db4-8fa9-77d296992e3d-kube-api-access-p5wgh\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:30.284603 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.284613    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/453b04b9-0a28-4db4-8fa9-77d296992e3d-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:30.284857 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.284623    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/453b04b9-0a28-4db4-8fa9-77d296992e3d-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:30.316856 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.316817    2527 generic.go:356] "Generic (PLEG): container finished" podID="453b04b9-0a28-4db4-8fa9-77d296992e3d" containerID="d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66" exitCode=0
Sep 10 15:59:30.317076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.316867    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"
Sep 10 15:59:30.317076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.316898    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb" event={"ID":"453b04b9-0a28-4db4-8fa9-77d296992e3d","Type":"ContainerDied","Data":"d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66"}
Sep 10 15:59:30.317076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.316955    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb" event={"ID":"453b04b9-0a28-4db4-8fa9-77d296992e3d","Type":"ContainerDied","Data":"c025987af31bc671565fd1437fe0daffecc319759439c7b6a11ae5cbbf69234e"}
Sep 10 15:59:30.317076 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.316982    2527 scope.go:117] "RemoveContainer" containerID="d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66"
Sep 10 15:59:30.327047 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.327027    2527 scope.go:117] "RemoveContainer" containerID="d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66"
Sep 10 15:59:30.327406 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:59:30.327386    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66\": container with ID starting with d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66 not found: ID does not exist" containerID="d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66"
Sep 10 15:59:30.327488 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.327419    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66"} err="failed to get container status \"d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66\": rpc error: code = NotFound desc = could not find container \"d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66\": container with ID starting with d04c1a9d2d0cde2e9635d1b475e0efb9ef3d0cdbfbeb466012117fb10793dd66 not found: ID does not exist"
Sep 10 15:59:30.339807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.339764    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"]
Sep 10 15:59:30.345643 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.345610    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-789be7-5ff5f8f64f-x56bb"]
Sep 10 15:59:30.564219 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:30.564096    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="453b04b9-0a28-4db4-8fa9-77d296992e3d" path="/var/lib/kubelet/pods/453b04b9-0a28-4db4-8fa9-77d296992e3d/volumes"
Sep 10 15:59:37.606502 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.606414    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"]
Sep 10 15:59:37.607052 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.607035    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="453b04b9-0a28-4db4-8fa9-77d296992e3d" containerName="maas-api"
Sep 10 15:59:37.607165 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.607054    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="453b04b9-0a28-4db4-8fa9-77d296992e3d" containerName="maas-api"
Sep 10 15:59:37.607165 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.607070    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a90c44c-a616-4cf6-a9b7-6f3b679408c7" containerName="maas-api"
Sep 10 15:59:37.607165 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.607080    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a90c44c-a616-4cf6-a9b7-6f3b679408c7" containerName="maas-api"
Sep 10 15:59:37.607165 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.607094    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceef1d38-ab9e-4963-b064-76e1e38f2d82" containerName="payload-pre-processing"
Sep 10 15:59:37.607165 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.607103    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceef1d38-ab9e-4963-b064-76e1e38f2d82" containerName="payload-pre-processing"
Sep 10 15:59:37.607429 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.607226    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="453b04b9-0a28-4db4-8fa9-77d296992e3d" containerName="maas-api"
Sep 10 15:59:37.607429 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.607238    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a90c44c-a616-4cf6-a9b7-6f3b679408c7" containerName="maas-api"
Sep 10 15:59:37.607429 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.607248    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceef1d38-ab9e-4963-b064-76e1e38f2d82" containerName="payload-pre-processing"
Sep 10 15:59:37.612147 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.612096    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.614518 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.614493    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-678f625f-serving-cert\""
Sep 10 15:59:37.625312 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.625279    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"]
Sep 10 15:59:37.650876 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.650838    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ba7779-cd4c-4876-b752-f91cae0cd627-openshift-service-ca\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.650876 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.650881    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-maas-api-tls\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.651156 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.650904    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-metrics-certs\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.651156 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.650960    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl7fz\" (UniqueName: \"kubernetes.io/projected/29ba7779-cd4c-4876-b752-f91cae0cd627-kube-api-access-sl7fz\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.752275 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.752230    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl7fz\" (UniqueName: \"kubernetes.io/projected/29ba7779-cd4c-4876-b752-f91cae0cd627-kube-api-access-sl7fz\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.752504 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.752343    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ba7779-cd4c-4876-b752-f91cae0cd627-openshift-service-ca\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.752504 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.752369    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-maas-api-tls\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.752504 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.752386    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-metrics-certs\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.752672 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:59:37.752501    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-serving-cert: secret "maas-api-e2e-mt-678f625f-serving-cert" not found
Sep 10 15:59:37.752672 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:59:37.752574    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-maas-api-tls podName:29ba7779-cd4c-4876-b752-f91cae0cd627 nodeName:}" failed. No retries permitted until 2026-09-10 15:59:38.252556209 +0000 UTC m=+1074.168681071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-maas-api-tls") pod "maas-api-e2e-mt-678f625f-54c66f4597-jbtkt" (UID: "29ba7779-cd4c-4876-b752-f91cae0cd627") : secret "maas-api-e2e-mt-678f625f-serving-cert" not found
Sep 10 15:59:37.753061 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.753037    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ba7779-cd4c-4876-b752-f91cae0cd627-openshift-service-ca\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.755028 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.755008    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-metrics-certs\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:37.763200 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:37.763164    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl7fz\" (UniqueName: \"kubernetes.io/projected/29ba7779-cd4c-4876-b752-f91cae0cd627-kube-api-access-sl7fz\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:38.257780 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:38.257735    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-maas-api-tls\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:38.260320 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:38.260296    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-maas-api-tls\") pod \"maas-api-e2e-mt-678f625f-54c66f4597-jbtkt\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:38.524722 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:38.524623    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:38.825342 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:38.825311    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"]
Sep 10 15:59:38.828755 ip-10-0-136-61 kubenswrapper[2527]: W0910 15:59:38.828728    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ba7779_cd4c_4876_b752_f91cae0cd627.slice/crio-a1fafdec5c4659a1a0c151d905c7cf889d9985bbbe00f0bd2ee032b8b2586e5e WatchSource:0}: Error finding container a1fafdec5c4659a1a0c151d905c7cf889d9985bbbe00f0bd2ee032b8b2586e5e: Status 404 returned error can't find the container with id a1fafdec5c4659a1a0c151d905c7cf889d9985bbbe00f0bd2ee032b8b2586e5e
Sep 10 15:59:39.376839 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:39.376796    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt" event={"ID":"29ba7779-cd4c-4876-b752-f91cae0cd627","Type":"ContainerStarted","Data":"dc39923d11446109735dde5fd71fc979a3cb39f998718d446ddb9b9dd154e763"}
Sep 10 15:59:39.377062 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:39.376875    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:39.377062 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:39.376902    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt" event={"ID":"29ba7779-cd4c-4876-b752-f91cae0cd627","Type":"ContainerStarted","Data":"a1fafdec5c4659a1a0c151d905c7cf889d9985bbbe00f0bd2ee032b8b2586e5e"}
Sep 10 15:59:39.404065 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:39.403994    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt" podStartSLOduration=1.995442988 podStartE2EDuration="2.403974342s" totalImagesPullingTime="408.531354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:59:39.402111421 +0000 UTC m=+1075.318236323" watchObservedRunningTime="2026-09-10 15:59:39.403974342 +0000 UTC m=+1075.320099221"
Sep 10 15:59:45.389293 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:45.389263    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 15:59:48.816613 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:48.816577    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"]
Sep 10 15:59:48.817112 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:48.816838    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk" podUID="bc5a112b-9a7c-4326-a00c-17b214f4a5fe" containerName="maas-api" containerID="cri-o://c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e" gracePeriod=30
Sep 10 15:59:48.897890 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:48.897850    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"]
Sep 10 15:59:48.898217 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:48.898127    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22" podUID="dda3fd75-5ec6-49f7-885f-9656380b924f" containerName="payload-pre-processing" containerID="cri-o://29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13" gracePeriod=30
Sep 10 15:59:49.161869 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.161841    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:59:49.192767 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.192732    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:59:49.261950 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.261911    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8pnq\" (UniqueName: \"kubernetes.io/projected/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-kube-api-access-g8pnq\") pod \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") "
Sep 10 15:59:49.262168 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.261961    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-openshift-service-ca\") pod \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") "
Sep 10 15:59:49.262168 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.261989    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-maas-api-tls\") pod \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") "
Sep 10 15:59:49.262168 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.262060    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-metrics-certs\") pod \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\" (UID: \"bc5a112b-9a7c-4326-a00c-17b214f4a5fe\") "
Sep 10 15:59:49.262168 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.262099    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htfjg\" (UniqueName: \"kubernetes.io/projected/dda3fd75-5ec6-49f7-885f-9656380b924f-kube-api-access-htfjg\") pod \"dda3fd75-5ec6-49f7-885f-9656380b924f\" (UID: \"dda3fd75-5ec6-49f7-885f-9656380b924f\") "
Sep 10 15:59:49.262168 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.262147    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda3fd75-5ec6-49f7-885f-9656380b924f-plugins-config-volume\") pod \"dda3fd75-5ec6-49f7-885f-9656380b924f\" (UID: \"dda3fd75-5ec6-49f7-885f-9656380b924f\") "
Sep 10 15:59:49.262569 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.262534    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bc5a112b-9a7c-4326-a00c-17b214f4a5fe" (UID: "bc5a112b-9a7c-4326-a00c-17b214f4a5fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:59:49.262670 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.262573    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda3fd75-5ec6-49f7-885f-9656380b924f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dda3fd75-5ec6-49f7-885f-9656380b924f" (UID: "dda3fd75-5ec6-49f7-885f-9656380b924f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:59:49.262792 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.262772    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda3fd75-5ec6-49f7-885f-9656380b924f-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:49.262792 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.262792    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:49.264533 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.264503    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-kube-api-access-g8pnq" (OuterVolumeSpecName: "kube-api-access-g8pnq") pod "bc5a112b-9a7c-4326-a00c-17b214f4a5fe" (UID: "bc5a112b-9a7c-4326-a00c-17b214f4a5fe"). InnerVolumeSpecName "kube-api-access-g8pnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:59:49.264824 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.264807    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bc5a112b-9a7c-4326-a00c-17b214f4a5fe" (UID: "bc5a112b-9a7c-4326-a00c-17b214f4a5fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:59:49.264900 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.264826    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda3fd75-5ec6-49f7-885f-9656380b924f-kube-api-access-htfjg" (OuterVolumeSpecName: "kube-api-access-htfjg") pod "dda3fd75-5ec6-49f7-885f-9656380b924f" (UID: "dda3fd75-5ec6-49f7-885f-9656380b924f"). InnerVolumeSpecName "kube-api-access-htfjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:59:49.264940 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.264918    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bc5a112b-9a7c-4326-a00c-17b214f4a5fe" (UID: "bc5a112b-9a7c-4326-a00c-17b214f4a5fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:59:49.363576 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.363482    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:49.363576 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.363523    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htfjg\" (UniqueName: \"kubernetes.io/projected/dda3fd75-5ec6-49f7-885f-9656380b924f-kube-api-access-htfjg\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:49.363576 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.363535    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8pnq\" (UniqueName: \"kubernetes.io/projected/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-kube-api-access-g8pnq\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:49.363576 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.363544    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc5a112b-9a7c-4326-a00c-17b214f4a5fe-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 15:59:49.433630 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.433592    2527 generic.go:356] "Generic (PLEG): container finished" podID="bc5a112b-9a7c-4326-a00c-17b214f4a5fe" containerID="c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e" exitCode=0
Sep 10 15:59:49.433807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.433640    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"
Sep 10 15:59:49.433807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.433679    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk" event={"ID":"bc5a112b-9a7c-4326-a00c-17b214f4a5fe","Type":"ContainerDied","Data":"c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e"}
Sep 10 15:59:49.433807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.433749    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk" event={"ID":"bc5a112b-9a7c-4326-a00c-17b214f4a5fe","Type":"ContainerDied","Data":"22929957946f19a232bd982e7a37a6e58c036a841fd570e9eaf3762fdea4fc34"}
Sep 10 15:59:49.433807 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.433779    2527 scope.go:117] "RemoveContainer" containerID="c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e"
Sep 10 15:59:49.435298 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.435272    2527 generic.go:356] "Generic (PLEG): container finished" podID="dda3fd75-5ec6-49f7-885f-9656380b924f" containerID="29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13" exitCode=0
Sep 10 15:59:49.435395 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.435314    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22" event={"ID":"dda3fd75-5ec6-49f7-885f-9656380b924f","Type":"ContainerDied","Data":"29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13"}
Sep 10 15:59:49.435395 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.435335    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"
Sep 10 15:59:49.435395 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.435352    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22" event={"ID":"dda3fd75-5ec6-49f7-885f-9656380b924f","Type":"ContainerDied","Data":"77040f36c101fd165007bc48079d5d79c7c3ecca3033fcfe51d56033b3ee1fc2"}
Sep 10 15:59:49.449416 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.449392    2527 scope.go:117] "RemoveContainer" containerID="c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e"
Sep 10 15:59:49.449819 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:59:49.449793    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e\": container with ID starting with c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e not found: ID does not exist" containerID="c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e"
Sep 10 15:59:49.449923 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.449826    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e"} err="failed to get container status \"c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e\": rpc error: code = NotFound desc = could not find container \"c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e\": container with ID starting with c6448a0960bedab8eceeecd51a8317e007a523291ce6647f6318aa53e1908f1e not found: ID does not exist"
Sep 10 15:59:49.449923 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.449847    2527 scope.go:117] "RemoveContainer" containerID="29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13"
Sep 10 15:59:49.458956 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.458931    2527 scope.go:117] "RemoveContainer" containerID="29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13"
Sep 10 15:59:49.459316 ip-10-0-136-61 kubenswrapper[2527]: E0910 15:59:49.459295    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13\": container with ID starting with 29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13 not found: ID does not exist" containerID="29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13"
Sep 10 15:59:49.459367 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.459328    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13"} err="failed to get container status \"29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13\": rpc error: code = NotFound desc = could not find container \"29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13\": container with ID starting with 29eb35c79e76380971e92cc039e4fd8c44a7bb905ce2bf13596cf1e714864d13 not found: ID does not exist"
Sep 10 15:59:49.477358 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.477316    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"]
Sep 10 15:59:49.484571 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.484538    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5d6f34-57699bd8fd-95g22"]
Sep 10 15:59:49.507560 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.507521    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"]
Sep 10 15:59:49.510009 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:49.509978    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d6f34-bb6d7b69d-jsrmk"]
Sep 10 15:59:50.564609 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:50.564571    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc5a112b-9a7c-4326-a00c-17b214f4a5fe" path="/var/lib/kubelet/pods/bc5a112b-9a7c-4326-a00c-17b214f4a5fe/volumes"
Sep 10 15:59:50.565159 ip-10-0-136-61 kubenswrapper[2527]: I0910 15:59:50.565115    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda3fd75-5ec6-49f7-885f-9656380b924f" path="/var/lib/kubelet/pods/dda3fd75-5ec6-49f7-885f-9656380b924f/volumes"
Sep 10 16:00:01.201271 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.201235    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"]
Sep 10 16:00:01.201769 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.201645    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5a112b-9a7c-4326-a00c-17b214f4a5fe" containerName="maas-api"
Sep 10 16:00:01.201769 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.201657    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5a112b-9a7c-4326-a00c-17b214f4a5fe" containerName="maas-api"
Sep 10 16:00:01.201769 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.201689    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda3fd75-5ec6-49f7-885f-9656380b924f" containerName="payload-pre-processing"
Sep 10 16:00:01.201769 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.201695    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda3fd75-5ec6-49f7-885f-9656380b924f" containerName="payload-pre-processing"
Sep 10 16:00:01.201922 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.201778    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda3fd75-5ec6-49f7-885f-9656380b924f" containerName="payload-pre-processing"
Sep 10 16:00:01.201922 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.201787    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5a112b-9a7c-4326-a00c-17b214f4a5fe" containerName="maas-api"
Sep 10 16:00:01.206963 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.206935    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.209395 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.209367    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-50ed73-serving-cert\""
Sep 10 16:00:01.222047 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.222011    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"]
Sep 10 16:00:01.280351 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.280316    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.280537 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.280400    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clr9r\" (UniqueName: \"kubernetes.io/projected/018cae23-2e24-4116-a13b-a9db57cdcd8e-kube-api-access-clr9r\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.280537 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.280431    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.280537 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.280455    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018cae23-2e24-4116-a13b-a9db57cdcd8e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.381608 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.381562    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clr9r\" (UniqueName: \"kubernetes.io/projected/018cae23-2e24-4116-a13b-a9db57cdcd8e-kube-api-access-clr9r\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.381608 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.381610    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.381842 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.381637    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018cae23-2e24-4116-a13b-a9db57cdcd8e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.381842 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.381673    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.381842 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:00:01.381750    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-serving-cert: secret "maas-api-e2e-shared-a-w3-50ed73-serving-cert" not found
Sep 10 16:00:01.381969 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:00:01.381852    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-maas-api-tls podName:018cae23-2e24-4116-a13b-a9db57cdcd8e nodeName:}" failed. No retries permitted until 2026-09-10 16:00:01.881828414 +0000 UTC m=+1097.797953278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-maas-api-tls") pod "maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6" (UID: "018cae23-2e24-4116-a13b-a9db57cdcd8e") : secret "maas-api-e2e-shared-a-w3-50ed73-serving-cert" not found
Sep 10 16:00:01.382445 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.382419    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018cae23-2e24-4116-a13b-a9db57cdcd8e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.384301 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.384270    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.390914 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.390885    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clr9r\" (UniqueName: \"kubernetes.io/projected/018cae23-2e24-4116-a13b-a9db57cdcd8e-kube-api-access-clr9r\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.887098 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.887063    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:01.889833 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:01.889798    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:02.121468 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:02.121429    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:02.407637 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:02.407609    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"]
Sep 10 16:00:02.410042 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:00:02.410011    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018cae23_2e24_4116_a13b_a9db57cdcd8e.slice/crio-2bfc0f1a6c38a8e4e721925cbd19b61ce326321a1fc50eb58f4597cdcd69b84e WatchSource:0}: Error finding container 2bfc0f1a6c38a8e4e721925cbd19b61ce326321a1fc50eb58f4597cdcd69b84e: Status 404 returned error can't find the container with id 2bfc0f1a6c38a8e4e721925cbd19b61ce326321a1fc50eb58f4597cdcd69b84e
Sep 10 16:00:02.510490 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:02.510401    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6" event={"ID":"018cae23-2e24-4116-a13b-a9db57cdcd8e","Type":"ContainerStarted","Data":"2bfc0f1a6c38a8e4e721925cbd19b61ce326321a1fc50eb58f4597cdcd69b84e"}
Sep 10 16:00:04.522112 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:04.522072    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6" event={"ID":"018cae23-2e24-4116-a13b-a9db57cdcd8e","Type":"ContainerStarted","Data":"edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc"}
Sep 10 16:00:04.522538 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:04.522280    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:04.547029 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:04.546970    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6" podStartSLOduration=2.312950985 podStartE2EDuration="3.546951987s" totalImagesPullingTime="1.234001002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:00:04.545453215 +0000 UTC m=+1100.461578105" watchObservedRunningTime="2026-09-10 16:00:04.546951987 +0000 UTC m=+1100.463076866"
Sep 10 16:00:10.536333 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:10.536303    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:00:20.488845 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.488803    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"]
Sep 10 16:00:20.489726 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.489686    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt" podUID="29ba7779-cd4c-4876-b752-f91cae0cd627" containerName="maas-api" containerID="cri-o://dc39923d11446109735dde5fd71fc979a3cb39f998718d446ddb9b9dd154e763" gracePeriod=30
Sep 10 16:00:20.611219 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.611185    2527 generic.go:356] "Generic (PLEG): container finished" podID="29ba7779-cd4c-4876-b752-f91cae0cd627" containerID="dc39923d11446109735dde5fd71fc979a3cb39f998718d446ddb9b9dd154e763" exitCode=0
Sep 10 16:00:20.611448 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.611242    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt" event={"ID":"29ba7779-cd4c-4876-b752-f91cae0cd627","Type":"ContainerDied","Data":"dc39923d11446109735dde5fd71fc979a3cb39f998718d446ddb9b9dd154e763"}
Sep 10 16:00:20.751072 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.751001    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 16:00:20.763388 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.763347    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ba7779-cd4c-4876-b752-f91cae0cd627-openshift-service-ca\") pod \"29ba7779-cd4c-4876-b752-f91cae0cd627\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") "
Sep 10 16:00:20.763565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.763404    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-maas-api-tls\") pod \"29ba7779-cd4c-4876-b752-f91cae0cd627\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") "
Sep 10 16:00:20.763565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.763440    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl7fz\" (UniqueName: \"kubernetes.io/projected/29ba7779-cd4c-4876-b752-f91cae0cd627-kube-api-access-sl7fz\") pod \"29ba7779-cd4c-4876-b752-f91cae0cd627\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") "
Sep 10 16:00:20.763565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.763477    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-metrics-certs\") pod \"29ba7779-cd4c-4876-b752-f91cae0cd627\" (UID: \"29ba7779-cd4c-4876-b752-f91cae0cd627\") "
Sep 10 16:00:20.763820 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.763792    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29ba7779-cd4c-4876-b752-f91cae0cd627-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29ba7779-cd4c-4876-b752-f91cae0cd627" (UID: "29ba7779-cd4c-4876-b752-f91cae0cd627"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:00:20.765799 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.765766    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ba7779-cd4c-4876-b752-f91cae0cd627-kube-api-access-sl7fz" (OuterVolumeSpecName: "kube-api-access-sl7fz") pod "29ba7779-cd4c-4876-b752-f91cae0cd627" (UID: "29ba7779-cd4c-4876-b752-f91cae0cd627"). InnerVolumeSpecName "kube-api-access-sl7fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:00:20.766204 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.766176    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29ba7779-cd4c-4876-b752-f91cae0cd627" (UID: "29ba7779-cd4c-4876-b752-f91cae0cd627"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:00:20.766342 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.766170    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "29ba7779-cd4c-4876-b752-f91cae0cd627" (UID: "29ba7779-cd4c-4876-b752-f91cae0cd627"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:00:20.864951 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.864908    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ba7779-cd4c-4876-b752-f91cae0cd627-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:00:20.864951 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.864944    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:00:20.864951 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.864954    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl7fz\" (UniqueName: \"kubernetes.io/projected/29ba7779-cd4c-4876-b752-f91cae0cd627-kube-api-access-sl7fz\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:00:20.864951 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:20.864963    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ba7779-cd4c-4876-b752-f91cae0cd627-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:00:21.618862 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:21.618834    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"
Sep 10 16:00:21.618862 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:21.618851    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt" event={"ID":"29ba7779-cd4c-4876-b752-f91cae0cd627","Type":"ContainerDied","Data":"a1fafdec5c4659a1a0c151d905c7cf889d9985bbbe00f0bd2ee032b8b2586e5e"}
Sep 10 16:00:21.619399 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:21.618908    2527 scope.go:117] "RemoveContainer" containerID="dc39923d11446109735dde5fd71fc979a3cb39f998718d446ddb9b9dd154e763"
Sep 10 16:00:21.646247 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:21.646210    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"]
Sep 10 16:00:21.653489 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:21.653458    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-678f625f-54c66f4597-jbtkt"]
Sep 10 16:00:22.564369 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:22.564335    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29ba7779-cd4c-4876-b752-f91cae0cd627" path="/var/lib/kubelet/pods/29ba7779-cd4c-4876-b752-f91cae0cd627/volumes"
Sep 10 16:00:29.902048 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.902005    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"]
Sep 10 16:00:29.902693 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.902670    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ba7779-cd4c-4876-b752-f91cae0cd627" containerName="maas-api"
Sep 10 16:00:29.902740 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.902698    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ba7779-cd4c-4876-b752-f91cae0cd627" containerName="maas-api"
Sep 10 16:00:29.902839 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.902826    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ba7779-cd4c-4876-b752-f91cae0cd627" containerName="maas-api"
Sep 10 16:00:29.907945 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.907913    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:29.910456 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.910428    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-67df2b-serving-cert\""
Sep 10 16:00:29.918080 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.918048    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"]
Sep 10 16:00:29.950510 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.950467    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfnck\" (UniqueName: \"kubernetes.io/projected/5956aa90-2b25-4fef-af35-2bf5486f6ef8-kube-api-access-nfnck\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:29.950707 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.950599    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:29.950707 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.950654    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5956aa90-2b25-4fef-af35-2bf5486f6ef8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:29.950707 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:29.950690    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.051663 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.051620    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfnck\" (UniqueName: \"kubernetes.io/projected/5956aa90-2b25-4fef-af35-2bf5486f6ef8-kube-api-access-nfnck\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.051884 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.051717    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.051884 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.051765    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5956aa90-2b25-4fef-af35-2bf5486f6ef8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.051884 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.051821    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.052068 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:00:30.051885    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-serving-cert: secret "maas-api-e2e-shared-b-w3-67df2b-serving-cert" not found
Sep 10 16:00:30.052068 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:00:30.051984    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-maas-api-tls podName:5956aa90-2b25-4fef-af35-2bf5486f6ef8 nodeName:}" failed. No retries permitted until 2026-09-10 16:00:30.551958814 +0000 UTC m=+1126.468083686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-maas-api-tls") pod "maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb" (UID: "5956aa90-2b25-4fef-af35-2bf5486f6ef8") : secret "maas-api-e2e-shared-b-w3-67df2b-serving-cert" not found
Sep 10 16:00:30.052639 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.052593    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5956aa90-2b25-4fef-af35-2bf5486f6ef8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.054590 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.054568    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.070322 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.070288    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfnck\" (UniqueName: \"kubernetes.io/projected/5956aa90-2b25-4fef-af35-2bf5486f6ef8-kube-api-access-nfnck\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.555922 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.555879    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.558480 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.558454    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.742816 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.742773    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"]
Sep 10 16:00:30.746625 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.746601    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:30.750984 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.750956    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-67df2b\""
Sep 10 16:00:30.751158 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.751041    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-67df2b-dockercfg-xjgww\""
Sep 10 16:00:30.760381 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.760347    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"]
Sep 10 16:00:30.822737 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.822651    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:30.858241 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.858207    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deb3119-081a-4839-a4ef-f0f0c4868d44-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8\" (UID: \"3deb3119-081a-4839-a4ef-f0f0c4868d44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:30.858421 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.858337    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwdbx\" (UniqueName: \"kubernetes.io/projected/3deb3119-081a-4839-a4ef-f0f0c4868d44-kube-api-access-cwdbx\") pod \"payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8\" (UID: \"3deb3119-081a-4839-a4ef-f0f0c4868d44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:30.959276 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.959240    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deb3119-081a-4839-a4ef-f0f0c4868d44-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8\" (UID: \"3deb3119-081a-4839-a4ef-f0f0c4868d44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:30.959777 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.959380    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwdbx\" (UniqueName: \"kubernetes.io/projected/3deb3119-081a-4839-a4ef-f0f0c4868d44-kube-api-access-cwdbx\") pod \"payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8\" (UID: \"3deb3119-081a-4839-a4ef-f0f0c4868d44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:30.960168 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.960126    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deb3119-081a-4839-a4ef-f0f0c4868d44-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8\" (UID: \"3deb3119-081a-4839-a4ef-f0f0c4868d44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:30.973049 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:30.973018    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwdbx\" (UniqueName: \"kubernetes.io/projected/3deb3119-081a-4839-a4ef-f0f0c4868d44-kube-api-access-cwdbx\") pod \"payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8\" (UID: \"3deb3119-081a-4839-a4ef-f0f0c4868d44\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:31.057693 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.057662    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:31.107252 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.107224    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"]
Sep 10 16:00:31.109721 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:00:31.109678    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5956aa90_2b25_4fef_af35_2bf5486f6ef8.slice/crio-6644922840f1536edaac23078563ed1cab908cdff32d87de324dca42c2e34bed WatchSource:0}: Error finding container 6644922840f1536edaac23078563ed1cab908cdff32d87de324dca42c2e34bed: Status 404 returned error can't find the container with id 6644922840f1536edaac23078563ed1cab908cdff32d87de324dca42c2e34bed
Sep 10 16:00:31.111296 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.111273    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:00:31.344098 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.343951    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"]
Sep 10 16:00:31.347105 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:00:31.347074    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3deb3119_081a_4839_a4ef_f0f0c4868d44.slice/crio-9fcabaff0b7a3424ab3c8e7a47259182d3ba895bdd9671b91a07d596a3267d31 WatchSource:0}: Error finding container 9fcabaff0b7a3424ab3c8e7a47259182d3ba895bdd9671b91a07d596a3267d31: Status 404 returned error can't find the container with id 9fcabaff0b7a3424ab3c8e7a47259182d3ba895bdd9671b91a07d596a3267d31
Sep 10 16:00:31.677048 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.677007    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb" event={"ID":"5956aa90-2b25-4fef-af35-2bf5486f6ef8","Type":"ContainerStarted","Data":"5e271eae6401524d1ce9265f5052bc0c6b85e8db4672d7c6231c05bc301205a4"}
Sep 10 16:00:31.677276 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.677074    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb" event={"ID":"5956aa90-2b25-4fef-af35-2bf5486f6ef8","Type":"ContainerStarted","Data":"6644922840f1536edaac23078563ed1cab908cdff32d87de324dca42c2e34bed"}
Sep 10 16:00:31.677276 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.677164    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:31.678904 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.678868    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8" event={"ID":"3deb3119-081a-4839-a4ef-f0f0c4868d44","Type":"ContainerStarted","Data":"db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6"}
Sep 10 16:00:31.679043 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.678932    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8" event={"ID":"3deb3119-081a-4839-a4ef-f0f0c4868d44","Type":"ContainerStarted","Data":"9fcabaff0b7a3424ab3c8e7a47259182d3ba895bdd9671b91a07d596a3267d31"}
Sep 10 16:00:31.679043 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.678958    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:31.706009 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.705945    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb" podStartSLOduration=2.212584532 podStartE2EDuration="2.705923046s" totalImagesPullingTime="493.338514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:00:31.703161023 +0000 UTC m=+1127.619285910" watchObservedRunningTime="2026-09-10 16:00:31.705923046 +0000 UTC m=+1127.622047925"
Sep 10 16:00:31.729014 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:31.728956    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8" podStartSLOduration=1.728938307 podStartE2EDuration="1.728938307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:00:31.728313751 +0000 UTC m=+1127.644438651" watchObservedRunningTime="2026-09-10 16:00:31.728938307 +0000 UTC m=+1127.645063186"
Sep 10 16:00:37.690274 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:37.690241    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:00:42.687041 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:42.687005    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:00:47.215028 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.214986    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"]
Sep 10 16:00:47.218717 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.218696    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.221307 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:00:47.221262    2527 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" is forbidden: User \"system:node:ip-10-0-136-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-136-61.ec2.internal' and this object" podUID="3abbba77-b006-4d29-9c33-5426702cac07" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.221478 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:00:47.221369    2527 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-shared-a-w2-a361a9-serving-cert\" is forbidden: User \"system:node:ip-10-0-136-61.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-136-61.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-a361a9-serving-cert\"" type="*v1.Secret"
Sep 10 16:00:47.288009 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.287973    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"]
Sep 10 16:00:47.317270 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.317230    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.317482 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.317281    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abbba77-b006-4d29-9c33-5426702cac07-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.317482 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.317315    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqkjz\" (UniqueName: \"kubernetes.io/projected/3abbba77-b006-4d29-9c33-5426702cac07-kube-api-access-wqkjz\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.317482 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.317355    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.418606 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.418557    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.418834 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.418614    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abbba77-b006-4d29-9c33-5426702cac07-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.418834 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.418636    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqkjz\" (UniqueName: \"kubernetes.io/projected/3abbba77-b006-4d29-9c33-5426702cac07-kube-api-access-wqkjz\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.418834 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.418666    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.419382 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.419354    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abbba77-b006-4d29-9c33-5426702cac07-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.421498 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.421470    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:47.430852 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:47.430824    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqkjz\" (UniqueName: \"kubernetes.io/projected/3abbba77-b006-4d29-9c33-5426702cac07-kube-api-access-wqkjz\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:48.419104 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:00:48.419063    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-serving-cert: failed to sync secret cache: timed out waiting for the condition
Sep 10 16:00:48.419522 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:00:48.419203    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-maas-api-tls podName:3abbba77-b006-4d29-9c33-5426702cac07 nodeName:}" failed. No retries permitted until 2026-09-10 16:00:48.919177456 +0000 UTC m=+1144.835302336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-maas-api-tls") pod "maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz" (UID: "3abbba77-b006-4d29-9c33-5426702cac07") : failed to sync secret cache: timed out waiting for the condition
Sep 10 16:00:48.796747 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:48.796710    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-a361a9-serving-cert\""
Sep 10 16:00:48.934585 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:48.934538    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:48.937453 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:48.937425    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:49.030413 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:49.030365    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:49.323457 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:49.323426    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"]
Sep 10 16:00:49.325321 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:00:49.325293    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3abbba77_b006_4d29_9c33_5426702cac07.slice/crio-6164cca8eec41a9dd6dba5ac22c3d79018fe382292120f9617ea4503d679a8cd WatchSource:0}: Error finding container 6164cca8eec41a9dd6dba5ac22c3d79018fe382292120f9617ea4503d679a8cd: Status 404 returned error can't find the container with id 6164cca8eec41a9dd6dba5ac22c3d79018fe382292120f9617ea4503d679a8cd
Sep 10 16:00:49.783831 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:49.783794    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz" event={"ID":"3abbba77-b006-4d29-9c33-5426702cac07","Type":"ContainerStarted","Data":"6164cca8eec41a9dd6dba5ac22c3d79018fe382292120f9617ea4503d679a8cd"}
Sep 10 16:00:50.791029 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:50.790991    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz" event={"ID":"3abbba77-b006-4d29-9c33-5426702cac07","Type":"ContainerStarted","Data":"06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a"}
Sep 10 16:00:50.791518 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:50.791057    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:00:50.811161 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:50.811083    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz" podStartSLOduration=3.256831771 podStartE2EDuration="3.811066971s" totalImagesPullingTime="554.2352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:00:50.810072201 +0000 UTC m=+1146.726197079" watchObservedRunningTime="2026-09-10 16:00:50.811066971 +0000 UTC m=+1146.727191850"
Sep 10 16:00:56.802224 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:00:56.802188    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:01:16.397380 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.397289    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"]
Sep 10 16:01:16.401701 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.401674    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.404284 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.404252    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-33451c-serving-cert\""
Sep 10 16:01:16.417093 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.417061    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"]
Sep 10 16:01:16.487755 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.487703    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.488061 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.487832    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b3ab5b-def6-41f1-a7a7-2494419781ee-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.488061 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.487861    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.488061 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.487963    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8gzz\" (UniqueName: \"kubernetes.io/projected/47b3ab5b-def6-41f1-a7a7-2494419781ee-kube-api-access-q8gzz\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.589497 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.589457    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.589706 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.589514    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b3ab5b-def6-41f1-a7a7-2494419781ee-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.589706 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.589535    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.589706 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.589566    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8gzz\" (UniqueName: \"kubernetes.io/projected/47b3ab5b-def6-41f1-a7a7-2494419781ee-kube-api-access-q8gzz\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.590392 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.590357    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b3ab5b-def6-41f1-a7a7-2494419781ee-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.592246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.592221    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.592357 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.592282    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.599945 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.599921    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8gzz\" (UniqueName: \"kubernetes.io/projected/47b3ab5b-def6-41f1-a7a7-2494419781ee-kube-api-access-q8gzz\") pod \"maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:16.717641 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:16.717535    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:17.002791 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:17.002753    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"]
Sep 10 16:01:17.005689 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:01:17.005651    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b3ab5b_def6_41f1_a7a7_2494419781ee.slice/crio-2511565094aecc80e119111432e9b18a6269c2fb79fd7781d606cacd1538bc3e WatchSource:0}: Error finding container 2511565094aecc80e119111432e9b18a6269c2fb79fd7781d606cacd1538bc3e: Status 404 returned error can't find the container with id 2511565094aecc80e119111432e9b18a6269c2fb79fd7781d606cacd1538bc3e
Sep 10 16:01:17.940985 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:17.940942    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c" event={"ID":"47b3ab5b-def6-41f1-a7a7-2494419781ee","Type":"ContainerStarted","Data":"40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0"}
Sep 10 16:01:17.941509 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:17.941006    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:17.941509 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:17.941022    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c" event={"ID":"47b3ab5b-def6-41f1-a7a7-2494419781ee","Type":"ContainerStarted","Data":"2511565094aecc80e119111432e9b18a6269c2fb79fd7781d606cacd1538bc3e"}
Sep 10 16:01:17.972477 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:17.972369    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c" podStartSLOduration=1.261434389 podStartE2EDuration="1.972351452s" totalImagesPullingTime="710.917063ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:01:17.970103839 +0000 UTC m=+1173.886228746" watchObservedRunningTime="2026-09-10 16:01:17.972351452 +0000 UTC m=+1173.888476338"
Sep 10 16:01:23.952765 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:23.952730    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:01:44.587603 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:44.587570    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 16:01:44.597145 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:44.597097    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 16:01:54.437612 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.437567    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"]
Sep 10 16:01:54.442519 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.442492    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.445224 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.445198    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0e6e0dcc-serving-cert\""
Sep 10 16:01:54.452981 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.452943    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"]
Sep 10 16:01:54.461571 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.461531    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-metrics-certs\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.461897 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.461865    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8034215-3f6f-47b8-96cf-98c42d36e333-openshift-service-ca\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.462032 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.461967    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-maas-api-tls\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.462032 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.462000    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v4mr\" (UniqueName: \"kubernetes.io/projected/e8034215-3f6f-47b8-96cf-98c42d36e333-kube-api-access-9v4mr\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.562664 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.562632    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8034215-3f6f-47b8-96cf-98c42d36e333-openshift-service-ca\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.562883 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.562686    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-maas-api-tls\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.562883 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.562709    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v4mr\" (UniqueName: \"kubernetes.io/projected/e8034215-3f6f-47b8-96cf-98c42d36e333-kube-api-access-9v4mr\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.562883 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.562750    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-metrics-certs\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.562883 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:01:54.562853    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-serving-cert: secret "maas-api-e2e-mt-0e6e0dcc-serving-cert" not found
Sep 10 16:01:54.563114 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:01:54.562932    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-maas-api-tls podName:e8034215-3f6f-47b8-96cf-98c42d36e333 nodeName:}" failed. No retries permitted until 2026-09-10 16:01:55.062909776 +0000 UTC m=+1210.979034646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-maas-api-tls") pod "maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5" (UID: "e8034215-3f6f-47b8-96cf-98c42d36e333") : secret "maas-api-e2e-mt-0e6e0dcc-serving-cert" not found
Sep 10 16:01:54.563578 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.563549    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8034215-3f6f-47b8-96cf-98c42d36e333-openshift-service-ca\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.565540 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.565516    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-metrics-certs\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:54.573562 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:54.573530    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v4mr\" (UniqueName: \"kubernetes.io/projected/e8034215-3f6f-47b8-96cf-98c42d36e333-kube-api-access-9v4mr\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:55.068419 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.068384    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-maas-api-tls\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:55.071426 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.071395    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-maas-api-tls\") pod \"maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:55.249196 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.249105    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"]
Sep 10 16:01:55.253048 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.253017    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:01:55.255867 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.255842    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0e6e0dcc-dockercfg-n9wsr\""
Sep 10 16:01:55.256195 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.256179    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0e6e0dcc\""
Sep 10 16:01:55.276751 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.276720    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"]
Sep 10 16:01:55.356860 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.356762    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:55.372183 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.372145    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e75edf-2de4-40dd-aac8-eff9b1353cad-plugins-config-volume\") pod \"payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg\" (UID: \"92e75edf-2de4-40dd-aac8-eff9b1353cad\") " pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:01:55.372329 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.372284    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr2kf\" (UniqueName: \"kubernetes.io/projected/92e75edf-2de4-40dd-aac8-eff9b1353cad-kube-api-access-nr2kf\") pod \"payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg\" (UID: \"92e75edf-2de4-40dd-aac8-eff9b1353cad\") " pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:01:55.473399 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.473363    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr2kf\" (UniqueName: \"kubernetes.io/projected/92e75edf-2de4-40dd-aac8-eff9b1353cad-kube-api-access-nr2kf\") pod \"payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg\" (UID: \"92e75edf-2de4-40dd-aac8-eff9b1353cad\") " pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:01:55.473818 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.473447    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e75edf-2de4-40dd-aac8-eff9b1353cad-plugins-config-volume\") pod \"payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg\" (UID: \"92e75edf-2de4-40dd-aac8-eff9b1353cad\") " pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:01:55.474147 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.474116    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e75edf-2de4-40dd-aac8-eff9b1353cad-plugins-config-volume\") pod \"payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg\" (UID: \"92e75edf-2de4-40dd-aac8-eff9b1353cad\") " pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:01:55.483320 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.483293    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr2kf\" (UniqueName: \"kubernetes.io/projected/92e75edf-2de4-40dd-aac8-eff9b1353cad-kube-api-access-nr2kf\") pod \"payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg\" (UID: \"92e75edf-2de4-40dd-aac8-eff9b1353cad\") " pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:01:55.569226 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.569189    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:01:55.642580 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.642544    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"]
Sep 10 16:01:55.647080 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:01:55.647041    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8034215_3f6f_47b8_96cf_98c42d36e333.slice/crio-e73f310a8404c1b1c98d5a21fb3aae4524e44482bf0310c25e7cbee60f98d709 WatchSource:0}: Error finding container e73f310a8404c1b1c98d5a21fb3aae4524e44482bf0310c25e7cbee60f98d709: Status 404 returned error can't find the container with id e73f310a8404c1b1c98d5a21fb3aae4524e44482bf0310c25e7cbee60f98d709
Sep 10 16:01:55.853334 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:55.853301    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"]
Sep 10 16:01:55.854707 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:01:55.854676    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e75edf_2de4_40dd_aac8_eff9b1353cad.slice/crio-2b8cc8077c6371d1c6ce39ad85491763e362de48db4e58c229fa60469c2eceac WatchSource:0}: Error finding container 2b8cc8077c6371d1c6ce39ad85491763e362de48db4e58c229fa60469c2eceac: Status 404 returned error can't find the container with id 2b8cc8077c6371d1c6ce39ad85491763e362de48db4e58c229fa60469c2eceac
Sep 10 16:01:56.161556 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:56.161445    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg" event={"ID":"92e75edf-2de4-40dd-aac8-eff9b1353cad","Type":"ContainerStarted","Data":"548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e"}
Sep 10 16:01:56.161556 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:56.161508    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg" event={"ID":"92e75edf-2de4-40dd-aac8-eff9b1353cad","Type":"ContainerStarted","Data":"2b8cc8077c6371d1c6ce39ad85491763e362de48db4e58c229fa60469c2eceac"}
Sep 10 16:01:56.161786 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:56.161573    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:01:56.163776 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:56.163748    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5" event={"ID":"e8034215-3f6f-47b8-96cf-98c42d36e333","Type":"ContainerStarted","Data":"c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7"}
Sep 10 16:01:56.163917 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:56.163811    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:01:56.163917 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:56.163835    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5" event={"ID":"e8034215-3f6f-47b8-96cf-98c42d36e333","Type":"ContainerStarted","Data":"e73f310a8404c1b1c98d5a21fb3aae4524e44482bf0310c25e7cbee60f98d709"}
Sep 10 16:01:56.181584 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:56.181526    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg" podStartSLOduration=1.181508638 podStartE2EDuration="1.181508638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:01:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:01:56.179844138 +0000 UTC m=+1212.095969037" watchObservedRunningTime="2026-09-10 16:01:56.181508638 +0000 UTC m=+1212.097633545"
Sep 10 16:01:56.205507 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:01:56.205437    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5" podStartSLOduration=1.800444141 podStartE2EDuration="2.205409649s" totalImagesPullingTime="404.965508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:01:56.202243918 +0000 UTC m=+1212.118368813" watchObservedRunningTime="2026-09-10 16:01:56.205409649 +0000 UTC m=+1212.121534529"
Sep 10 16:02:02.176039 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:02:02.176008    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:02:07.172128 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:02:07.172093    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:03:33.918182 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:33.918119    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"]
Sep 10 16:03:33.918627 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:33.918383    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5" podUID="e8034215-3f6f-47b8-96cf-98c42d36e333" containerName="maas-api" containerID="cri-o://c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7" gracePeriod=30
Sep 10 16:03:33.976639 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:33.976597    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"]
Sep 10 16:03:33.976950 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:33.976890    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg" podUID="92e75edf-2de4-40dd-aac8-eff9b1353cad" containerName="payload-processing" containerID="cri-o://548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e" gracePeriod=30
Sep 10 16:03:34.269894 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.269865    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:03:34.273341 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.273313    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:03:34.360511 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.360468    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nr2kf\" (UniqueName: \"kubernetes.io/projected/92e75edf-2de4-40dd-aac8-eff9b1353cad-kube-api-access-nr2kf\") pod \"92e75edf-2de4-40dd-aac8-eff9b1353cad\" (UID: \"92e75edf-2de4-40dd-aac8-eff9b1353cad\") "
Sep 10 16:03:34.360511 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.360511    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-maas-api-tls\") pod \"e8034215-3f6f-47b8-96cf-98c42d36e333\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") "
Sep 10 16:03:34.360744 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.360609    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e75edf-2de4-40dd-aac8-eff9b1353cad-plugins-config-volume\") pod \"92e75edf-2de4-40dd-aac8-eff9b1353cad\" (UID: \"92e75edf-2de4-40dd-aac8-eff9b1353cad\") "
Sep 10 16:03:34.360744 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.360678    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-metrics-certs\") pod \"e8034215-3f6f-47b8-96cf-98c42d36e333\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") "
Sep 10 16:03:34.360744 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.360718    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8034215-3f6f-47b8-96cf-98c42d36e333-openshift-service-ca\") pod \"e8034215-3f6f-47b8-96cf-98c42d36e333\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") "
Sep 10 16:03:34.360901 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.360765    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v4mr\" (UniqueName: \"kubernetes.io/projected/e8034215-3f6f-47b8-96cf-98c42d36e333-kube-api-access-9v4mr\") pod \"e8034215-3f6f-47b8-96cf-98c42d36e333\" (UID: \"e8034215-3f6f-47b8-96cf-98c42d36e333\") "
Sep 10 16:03:34.361090 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.361054    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92e75edf-2de4-40dd-aac8-eff9b1353cad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "92e75edf-2de4-40dd-aac8-eff9b1353cad" (UID: "92e75edf-2de4-40dd-aac8-eff9b1353cad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:03:34.361238 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.361171    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8034215-3f6f-47b8-96cf-98c42d36e333-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e8034215-3f6f-47b8-96cf-98c42d36e333" (UID: "e8034215-3f6f-47b8-96cf-98c42d36e333"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:03:34.363011 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.362974    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e8034215-3f6f-47b8-96cf-98c42d36e333" (UID: "e8034215-3f6f-47b8-96cf-98c42d36e333"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:03:34.363011 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.362986    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92e75edf-2de4-40dd-aac8-eff9b1353cad-kube-api-access-nr2kf" (OuterVolumeSpecName: "kube-api-access-nr2kf") pod "92e75edf-2de4-40dd-aac8-eff9b1353cad" (UID: "92e75edf-2de4-40dd-aac8-eff9b1353cad"). InnerVolumeSpecName "kube-api-access-nr2kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:03:34.363261 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.363016    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e8034215-3f6f-47b8-96cf-98c42d36e333" (UID: "e8034215-3f6f-47b8-96cf-98c42d36e333"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:03:34.363261 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.363069    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8034215-3f6f-47b8-96cf-98c42d36e333-kube-api-access-9v4mr" (OuterVolumeSpecName: "kube-api-access-9v4mr") pod "e8034215-3f6f-47b8-96cf-98c42d36e333" (UID: "e8034215-3f6f-47b8-96cf-98c42d36e333"). InnerVolumeSpecName "kube-api-access-9v4mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:03:34.462565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.462466    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr2kf\" (UniqueName: \"kubernetes.io/projected/92e75edf-2de4-40dd-aac8-eff9b1353cad-kube-api-access-nr2kf\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:03:34.462565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.462501    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:03:34.462565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.462510    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92e75edf-2de4-40dd-aac8-eff9b1353cad-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:03:34.462565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.462520    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8034215-3f6f-47b8-96cf-98c42d36e333-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:03:34.462565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.462530    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8034215-3f6f-47b8-96cf-98c42d36e333-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:03:34.462565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.462538    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v4mr\" (UniqueName: \"kubernetes.io/projected/e8034215-3f6f-47b8-96cf-98c42d36e333-kube-api-access-9v4mr\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:03:34.719793 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.719704    2527 generic.go:356] "Generic (PLEG): container finished" podID="e8034215-3f6f-47b8-96cf-98c42d36e333" containerID="c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7" exitCode=0
Sep 10 16:03:34.719793 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.719756    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"
Sep 10 16:03:34.719793 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.719781    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5" event={"ID":"e8034215-3f6f-47b8-96cf-98c42d36e333","Type":"ContainerDied","Data":"c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7"}
Sep 10 16:03:34.720075 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.719837    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5" event={"ID":"e8034215-3f6f-47b8-96cf-98c42d36e333","Type":"ContainerDied","Data":"e73f310a8404c1b1c98d5a21fb3aae4524e44482bf0310c25e7cbee60f98d709"}
Sep 10 16:03:34.720075 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.719860    2527 scope.go:117] "RemoveContainer" containerID="c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7"
Sep 10 16:03:34.721468 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.721443    2527 generic.go:356] "Generic (PLEG): container finished" podID="92e75edf-2de4-40dd-aac8-eff9b1353cad" containerID="548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e" exitCode=0
Sep 10 16:03:34.721569 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.721487    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"
Sep 10 16:03:34.721569 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.721490    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg" event={"ID":"92e75edf-2de4-40dd-aac8-eff9b1353cad","Type":"ContainerDied","Data":"548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e"}
Sep 10 16:03:34.721654 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.721605    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg" event={"ID":"92e75edf-2de4-40dd-aac8-eff9b1353cad","Type":"ContainerDied","Data":"2b8cc8077c6371d1c6ce39ad85491763e362de48db4e58c229fa60469c2eceac"}
Sep 10 16:03:34.730001 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.729976    2527 scope.go:117] "RemoveContainer" containerID="c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7"
Sep 10 16:03:34.730339 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:03:34.730315    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7\": container with ID starting with c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7 not found: ID does not exist" containerID="c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7"
Sep 10 16:03:34.730445 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.730345    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7"} err="failed to get container status \"c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7\": rpc error: code = NotFound desc = could not find container \"c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7\": container with ID starting with c65e6c1b6f366832117bd9504a5428f3cec68efc6e302262ed96cffbd158fed7 not found: ID does not exist"
Sep 10 16:03:34.730445 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.730362    2527 scope.go:117] "RemoveContainer" containerID="548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e"
Sep 10 16:03:34.739367 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.739346    2527 scope.go:117] "RemoveContainer" containerID="548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e"
Sep 10 16:03:34.739711 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:03:34.739689    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e\": container with ID starting with 548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e not found: ID does not exist" containerID="548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e"
Sep 10 16:03:34.739780 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.739722    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e"} err="failed to get container status \"548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e\": rpc error: code = NotFound desc = could not find container \"548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e\": container with ID starting with 548728735bdd2648211293ae7ec9644f5cf788e6020c466f9b77d91ba855277e not found: ID does not exist"
Sep 10 16:03:34.747152 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.747104    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"]
Sep 10 16:03:34.752353 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.752319    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e6e0dcc-5dfd69cd5f-22jh5"]
Sep 10 16:03:34.763552 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.763520    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"]
Sep 10 16:03:34.767851 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:34.767824    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0e6e0dcc-65dc78688f-48brg"]
Sep 10 16:03:36.565043 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:36.565007    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92e75edf-2de4-40dd-aac8-eff9b1353cad" path="/var/lib/kubelet/pods/92e75edf-2de4-40dd-aac8-eff9b1353cad/volumes"
Sep 10 16:03:36.565487 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:03:36.565472    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8034215-3f6f-47b8-96cf-98c42d36e333" path="/var/lib/kubelet/pods/e8034215-3f6f-47b8-96cf-98c42d36e333/volumes"
Sep 10 16:05:53.116042 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.115998    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"]
Sep 10 16:05:53.116686 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.116670    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8034215-3f6f-47b8-96cf-98c42d36e333" containerName="maas-api"
Sep 10 16:05:53.116752 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.116690    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8034215-3f6f-47b8-96cf-98c42d36e333" containerName="maas-api"
Sep 10 16:05:53.116752 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.116713    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e75edf-2de4-40dd-aac8-eff9b1353cad" containerName="payload-processing"
Sep 10 16:05:53.116752 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.116723    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e75edf-2de4-40dd-aac8-eff9b1353cad" containerName="payload-processing"
Sep 10 16:05:53.116852 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.116808    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e75edf-2de4-40dd-aac8-eff9b1353cad" containerName="payload-processing"
Sep 10 16:05:53.116852 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.116825    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8034215-3f6f-47b8-96cf-98c42d36e333" containerName="maas-api"
Sep 10 16:05:53.121407 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.120637    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.123580 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.123548    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-45e544-serving-cert\""
Sep 10 16:05:53.130998 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.130962    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkgqn\" (UniqueName: \"kubernetes.io/projected/aa607248-3f98-46af-b0ff-8d5d0042f39d-kube-api-access-gkgqn\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.131168 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.131002    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa607248-3f98-46af-b0ff-8d5d0042f39d-openshift-service-ca\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.131168 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.131075    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-maas-api-tls\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.131301 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.131245    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-metrics-certs\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.138957 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.138922    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"]
Sep 10 16:05:53.231682 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.231643    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-metrics-certs\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.231908 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.231692    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkgqn\" (UniqueName: \"kubernetes.io/projected/aa607248-3f98-46af-b0ff-8d5d0042f39d-kube-api-access-gkgqn\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.231908 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.231717    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa607248-3f98-46af-b0ff-8d5d0042f39d-openshift-service-ca\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.231908 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.231751    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-maas-api-tls\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.231908 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:05:53.231870    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-serving-cert: secret "maas-api-e2e-api-a-45e544-serving-cert" not found
Sep 10 16:05:53.232177 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:05:53.231942    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-maas-api-tls podName:aa607248-3f98-46af-b0ff-8d5d0042f39d nodeName:}" failed. No retries permitted until 2026-09-10 16:05:53.731922435 +0000 UTC m=+1449.648047292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-maas-api-tls") pod "maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8" (UID: "aa607248-3f98-46af-b0ff-8d5d0042f39d") : secret "maas-api-e2e-api-a-45e544-serving-cert" not found
Sep 10 16:05:53.232601 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.232574    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa607248-3f98-46af-b0ff-8d5d0042f39d-openshift-service-ca\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.234665 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.234640    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-metrics-certs\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.241273 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.241247    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkgqn\" (UniqueName: \"kubernetes.io/projected/aa607248-3f98-46af-b0ff-8d5d0042f39d-kube-api-access-gkgqn\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.737261 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.737199    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-maas-api-tls\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.740193 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.740155    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-maas-api-tls\") pod \"maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:53.994494 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.994400    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"]
Sep 10 16:05:53.998458 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:53.998428    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:05:54.000821 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.000793    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-45e544\""
Sep 10 16:05:54.000974 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.000912    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-45e544-dockercfg-q7kgz\""
Sep 10 16:05:54.027100 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.027066    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"]
Sep 10 16:05:54.032661 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.032630    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:05:54.042422 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.042382    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcw9h\" (UniqueName: \"kubernetes.io/projected/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-kube-api-access-mcw9h\") pod \"payload-processing-e2e-api-a-45e544-77484975f7-m26g9\" (UID: \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:05:54.042602 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.042573    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-45e544-77484975f7-m26g9\" (UID: \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:05:54.143481 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.143443    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-45e544-77484975f7-m26g9\" (UID: \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:05:54.143863 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.143504    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcw9h\" (UniqueName: \"kubernetes.io/projected/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-kube-api-access-mcw9h\") pod \"payload-processing-e2e-api-a-45e544-77484975f7-m26g9\" (UID: \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:05:54.144173 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.144152    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-45e544-77484975f7-m26g9\" (UID: \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:05:54.159777 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.159743    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcw9h\" (UniqueName: \"kubernetes.io/projected/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-kube-api-access-mcw9h\") pod \"payload-processing-e2e-api-a-45e544-77484975f7-m26g9\" (UID: \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:05:54.311201 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.311163    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:05:54.320893 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.320780    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"]
Sep 10 16:05:54.324965 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:05:54.324921    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa607248_3f98_46af_b0ff_8d5d0042f39d.slice/crio-fc67467441f28f1a012619ba4c6de1d917b6ec7b918029c99c3c129ca41e46c4 WatchSource:0}: Error finding container fc67467441f28f1a012619ba4c6de1d917b6ec7b918029c99c3c129ca41e46c4: Status 404 returned error can't find the container with id fc67467441f28f1a012619ba4c6de1d917b6ec7b918029c99c3c129ca41e46c4
Sep 10 16:05:54.326683 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.326660    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:05:54.484223 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.484188    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8" event={"ID":"aa607248-3f98-46af-b0ff-8d5d0042f39d","Type":"ContainerStarted","Data":"fc67467441f28f1a012619ba4c6de1d917b6ec7b918029c99c3c129ca41e46c4"}
Sep 10 16:05:54.595791 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:54.595755    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"]
Sep 10 16:05:54.597312 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:05:54.597275    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1452174_5b89_4702_aa7a_0fadbe5ffc3a.slice/crio-0d5375346732a6188ba6205a6b2c71f36f1e79dcfbef8964ac4ccc11bfa386ff WatchSource:0}: Error finding container 0d5375346732a6188ba6205a6b2c71f36f1e79dcfbef8964ac4ccc11bfa386ff: Status 404 returned error can't find the container with id 0d5375346732a6188ba6205a6b2c71f36f1e79dcfbef8964ac4ccc11bfa386ff
Sep 10 16:05:55.491859 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:55.491816    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9" event={"ID":"b1452174-5b89-4702-aa7a-0fadbe5ffc3a","Type":"ContainerStarted","Data":"868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c"}
Sep 10 16:05:55.492346 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:55.491882    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9" event={"ID":"b1452174-5b89-4702-aa7a-0fadbe5ffc3a","Type":"ContainerStarted","Data":"0d5375346732a6188ba6205a6b2c71f36f1e79dcfbef8964ac4ccc11bfa386ff"}
Sep 10 16:05:55.492346 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:55.491999    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:05:55.549867 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:55.549811    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9" podStartSLOduration=2.549794609 podStartE2EDuration="2.549794609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:05:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:05:55.546655049 +0000 UTC m=+1451.462779929" watchObservedRunningTime="2026-09-10 16:05:55.549794609 +0000 UTC m=+1451.465919487"
Sep 10 16:05:56.499258 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:56.499159    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8" event={"ID":"aa607248-3f98-46af-b0ff-8d5d0042f39d","Type":"ContainerStarted","Data":"fa632e3e973de67d1b68235a699ca71b50acd4d19760600ec69551a779e1bb11"}
Sep 10 16:05:56.524990 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:56.524927    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8" podStartSLOduration=1.651786738 podStartE2EDuration="3.524906703s" totalImagesPullingTime="1.873119965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:05:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:05:56.523073105 +0000 UTC m=+1452.439197980" watchObservedRunningTime="2026-09-10 16:05:56.524906703 +0000 UTC m=+1452.441031604"
Sep 10 16:05:57.504758 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:05:57.504720    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:06:03.517083 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:03.517050    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:06:06.502100 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:06.502069    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:06:22.200803 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.200751    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"]
Sep 10 16:06:22.205235 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.205205    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.207816 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.207788    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-81d3ea-serving-cert\""
Sep 10 16:06:22.221078 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.221046    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"]
Sep 10 16:06:22.328089 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.328026    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-metrics-certs\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.328320 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.328105    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c92k7\" (UniqueName: \"kubernetes.io/projected/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-kube-api-access-c92k7\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.328320 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.328201    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-maas-api-tls\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.328320 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.328247    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-openshift-service-ca\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.429885 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.429830    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-metrics-certs\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.430088 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.429899    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c92k7\" (UniqueName: \"kubernetes.io/projected/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-kube-api-access-c92k7\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.430088 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.429948    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-maas-api-tls\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.430088 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:06:22.430048    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-serving-cert: secret "maas-api-e2e-api-b-81d3ea-serving-cert" not found
Sep 10 16:06:22.430268 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.430100    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-openshift-service-ca\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.430268 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:06:22.430112    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-maas-api-tls podName:1aa11e5e-a478-4d74-a6ad-e063f6b6efd6 nodeName:}" failed. No retries permitted until 2026-09-10 16:06:22.930092438 +0000 UTC m=+1478.846217295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-maas-api-tls") pod "maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5" (UID: "1aa11e5e-a478-4d74-a6ad-e063f6b6efd6") : secret "maas-api-e2e-api-b-81d3ea-serving-cert" not found
Sep 10 16:06:22.430817 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.430795    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-openshift-service-ca\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.432581 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.432557    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-metrics-certs\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.444837 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.444810    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c92k7\" (UniqueName: \"kubernetes.io/projected/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-kube-api-access-c92k7\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.935900 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.935865    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-maas-api-tls\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:22.938903 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:22.938864    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-maas-api-tls\") pod \"maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:23.119472 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:23.119432    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:23.408802 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:23.408768    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"]
Sep 10 16:06:23.410856 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:06:23.410828    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa11e5e_a478_4d74_a6ad_e063f6b6efd6.slice/crio-e2127313e9c48694c358fa6cd289e600eba6483a9b0c1fdb19c797b9f83ac4a1 WatchSource:0}: Error finding container e2127313e9c48694c358fa6cd289e600eba6483a9b0c1fdb19c797b9f83ac4a1: Status 404 returned error can't find the container with id e2127313e9c48694c358fa6cd289e600eba6483a9b0c1fdb19c797b9f83ac4a1
Sep 10 16:06:23.650923 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:23.650887    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5" event={"ID":"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6","Type":"ContainerStarted","Data":"e2127313e9c48694c358fa6cd289e600eba6483a9b0c1fdb19c797b9f83ac4a1"}
Sep 10 16:06:25.662833 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:25.662798    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5" event={"ID":"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6","Type":"ContainerStarted","Data":"9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e"}
Sep 10 16:06:25.663325 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:25.662930    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:25.685258 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:25.685200    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5" podStartSLOduration=2.360397608 podStartE2EDuration="3.685181918s" totalImagesPullingTime="1.32478431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:06:25.68307981 +0000 UTC m=+1481.599204710" watchObservedRunningTime="2026-09-10 16:06:25.685181918 +0000 UTC m=+1481.601306798"
Sep 10 16:06:31.673961 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:31.673923    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:06:44.633239 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:44.633202    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 16:06:44.642940 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:06:44.642915    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 16:07:04.707425 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:04.707387    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"]
Sep 10 16:07:04.707872 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:04.707622    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8" podUID="aa607248-3f98-46af-b0ff-8d5d0042f39d" containerName="maas-api" containerID="cri-o://fa632e3e973de67d1b68235a699ca71b50acd4d19760600ec69551a779e1bb11" gracePeriod=30
Sep 10 16:07:04.785490 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:04.785441    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"]
Sep 10 16:07:04.786059 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:04.786019    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9" podUID="b1452174-5b89-4702-aa7a-0fadbe5ffc3a" containerName="payload-processing" containerID="cri-o://868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c" gracePeriod=30
Sep 10 16:07:04.878110 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:04.878073    2527 generic.go:356] "Generic (PLEG): container finished" podID="aa607248-3f98-46af-b0ff-8d5d0042f39d" containerID="fa632e3e973de67d1b68235a699ca71b50acd4d19760600ec69551a779e1bb11" exitCode=0
Sep 10 16:07:04.878300 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:04.878162    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8" event={"ID":"aa607248-3f98-46af-b0ff-8d5d0042f39d","Type":"ContainerDied","Data":"fa632e3e973de67d1b68235a699ca71b50acd4d19760600ec69551a779e1bb11"}
Sep 10 16:07:05.050052 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.050015    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:07:05.087383 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.087358    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:07:05.158262 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.158221    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-metrics-certs\") pod \"aa607248-3f98-46af-b0ff-8d5d0042f39d\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") "
Sep 10 16:07:05.158464 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.158281    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa607248-3f98-46af-b0ff-8d5d0042f39d-openshift-service-ca\") pod \"aa607248-3f98-46af-b0ff-8d5d0042f39d\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") "
Sep 10 16:07:05.158464 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.158383    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkgqn\" (UniqueName: \"kubernetes.io/projected/aa607248-3f98-46af-b0ff-8d5d0042f39d-kube-api-access-gkgqn\") pod \"aa607248-3f98-46af-b0ff-8d5d0042f39d\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") "
Sep 10 16:07:05.158464 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.158452    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-maas-api-tls\") pod \"aa607248-3f98-46af-b0ff-8d5d0042f39d\" (UID: \"aa607248-3f98-46af-b0ff-8d5d0042f39d\") "
Sep 10 16:07:05.158773 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.158721    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa607248-3f98-46af-b0ff-8d5d0042f39d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa607248-3f98-46af-b0ff-8d5d0042f39d" (UID: "aa607248-3f98-46af-b0ff-8d5d0042f39d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:07:05.160684 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.160647    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa607248-3f98-46af-b0ff-8d5d0042f39d" (UID: "aa607248-3f98-46af-b0ff-8d5d0042f39d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:07:05.160684 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.160666    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aa607248-3f98-46af-b0ff-8d5d0042f39d" (UID: "aa607248-3f98-46af-b0ff-8d5d0042f39d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:07:05.160832 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.160664    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa607248-3f98-46af-b0ff-8d5d0042f39d-kube-api-access-gkgqn" (OuterVolumeSpecName: "kube-api-access-gkgqn") pod "aa607248-3f98-46af-b0ff-8d5d0042f39d" (UID: "aa607248-3f98-46af-b0ff-8d5d0042f39d"). InnerVolumeSpecName "kube-api-access-gkgqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:07:05.260009 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.259905    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcw9h\" (UniqueName: \"kubernetes.io/projected/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-kube-api-access-mcw9h\") pod \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\" (UID: \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\") "
Sep 10 16:07:05.260009 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.259968    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-plugins-config-volume\") pod \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\" (UID: \"b1452174-5b89-4702-aa7a-0fadbe5ffc3a\") "
Sep 10 16:07:05.260339 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.260325    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkgqn\" (UniqueName: \"kubernetes.io/projected/aa607248-3f98-46af-b0ff-8d5d0042f39d-kube-api-access-gkgqn\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:05.260389 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.260344    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:05.260389 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.260354    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa607248-3f98-46af-b0ff-8d5d0042f39d-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:05.260389 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.260362    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa607248-3f98-46af-b0ff-8d5d0042f39d-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:05.260389 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.260366    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b1452174-5b89-4702-aa7a-0fadbe5ffc3a" (UID: "b1452174-5b89-4702-aa7a-0fadbe5ffc3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:07:05.262187 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.262160    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-kube-api-access-mcw9h" (OuterVolumeSpecName: "kube-api-access-mcw9h") pod "b1452174-5b89-4702-aa7a-0fadbe5ffc3a" (UID: "b1452174-5b89-4702-aa7a-0fadbe5ffc3a"). InnerVolumeSpecName "kube-api-access-mcw9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:07:05.361363 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.361318    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcw9h\" (UniqueName: \"kubernetes.io/projected/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-kube-api-access-mcw9h\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:05.361363 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.361356    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1452174-5b89-4702-aa7a-0fadbe5ffc3a-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:05.885154 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.885092    2527 generic.go:356] "Generic (PLEG): container finished" podID="b1452174-5b89-4702-aa7a-0fadbe5ffc3a" containerID="868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c" exitCode=0
Sep 10 16:07:05.885593 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.885175    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"
Sep 10 16:07:05.885593 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.885174    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9" event={"ID":"b1452174-5b89-4702-aa7a-0fadbe5ffc3a","Type":"ContainerDied","Data":"868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c"}
Sep 10 16:07:05.885593 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.885239    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9" event={"ID":"b1452174-5b89-4702-aa7a-0fadbe5ffc3a","Type":"ContainerDied","Data":"0d5375346732a6188ba6205a6b2c71f36f1e79dcfbef8964ac4ccc11bfa386ff"}
Sep 10 16:07:05.885593 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.885267    2527 scope.go:117] "RemoveContainer" containerID="868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c"
Sep 10 16:07:05.887063 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.887037    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8" event={"ID":"aa607248-3f98-46af-b0ff-8d5d0042f39d","Type":"ContainerDied","Data":"fc67467441f28f1a012619ba4c6de1d917b6ec7b918029c99c3c129ca41e46c4"}
Sep 10 16:07:05.887201 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.887090    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"
Sep 10 16:07:05.895322 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.895284    2527 scope.go:117] "RemoveContainer" containerID="868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c"
Sep 10 16:07:05.895674 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:07:05.895651    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c\": container with ID starting with 868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c not found: ID does not exist" containerID="868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c"
Sep 10 16:07:05.895766 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.895687    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c"} err="failed to get container status \"868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c\": rpc error: code = NotFound desc = could not find container \"868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c\": container with ID starting with 868fc6a1cf1c46d971940563c4211a8090e7cb490d8d6f7035710e0680f0406c not found: ID does not exist"
Sep 10 16:07:05.895766 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.895714    2527 scope.go:117] "RemoveContainer" containerID="fa632e3e973de67d1b68235a699ca71b50acd4d19760600ec69551a779e1bb11"
Sep 10 16:07:05.913907 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.913874    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"]
Sep 10 16:07:05.918878 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.918846    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45e544-5887d9d9df-ppgj8"]
Sep 10 16:07:05.934151 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.934089    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"]
Sep 10 16:07:05.936290 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:05.936265    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-45e544-77484975f7-m26g9"]
Sep 10 16:07:06.565062 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:06.564973    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa607248-3f98-46af-b0ff-8d5d0042f39d" path="/var/lib/kubelet/pods/aa607248-3f98-46af-b0ff-8d5d0042f39d/volumes"
Sep 10 16:07:06.565447 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:06.565430    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1452174-5b89-4702-aa7a-0fadbe5ffc3a" path="/var/lib/kubelet/pods/b1452174-5b89-4702-aa7a-0fadbe5ffc3a/volumes"
Sep 10 16:07:23.602785 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.602745    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"]
Sep 10 16:07:23.603485 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.603457    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1452174-5b89-4702-aa7a-0fadbe5ffc3a" containerName="payload-processing"
Sep 10 16:07:23.603485 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.603485    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1452174-5b89-4702-aa7a-0fadbe5ffc3a" containerName="payload-processing"
Sep 10 16:07:23.603693 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.603520    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa607248-3f98-46af-b0ff-8d5d0042f39d" containerName="maas-api"
Sep 10 16:07:23.603693 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.603530    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa607248-3f98-46af-b0ff-8d5d0042f39d" containerName="maas-api"
Sep 10 16:07:23.603693 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.603661    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa607248-3f98-46af-b0ff-8d5d0042f39d" containerName="maas-api"
Sep 10 16:07:23.603693 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.603680    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1452174-5b89-4702-aa7a-0fadbe5ffc3a" containerName="payload-processing"
Sep 10 16:07:23.607531 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.607504    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.611096 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.610894    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-c0ec41-serving-cert\""
Sep 10 16:07:23.620259 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.620229    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"]
Sep 10 16:07:23.646536 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.646502    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-metrics-certs\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.646738 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.646562    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rdht\" (UniqueName: \"kubernetes.io/projected/21e5382c-ab31-4fd4-a28d-e0823c47220d-kube-api-access-9rdht\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.646738 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.646711    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e5382c-ab31-4fd4-a28d-e0823c47220d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.646862 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.646768    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.747912 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.747876    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.748244 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.747932    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-metrics-certs\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.748244 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.747974    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rdht\" (UniqueName: \"kubernetes.io/projected/21e5382c-ab31-4fd4-a28d-e0823c47220d-kube-api-access-9rdht\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.748244 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:07:23.748069    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-serving-cert: secret "maas-api-e2e-ipp-a-c0ec41-serving-cert" not found
Sep 10 16:07:23.748244 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.748074    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e5382c-ab31-4fd4-a28d-e0823c47220d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.748244 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:07:23.748220    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-maas-api-tls podName:21e5382c-ab31-4fd4-a28d-e0823c47220d nodeName:}" failed. No retries permitted until 2026-09-10 16:07:24.248195817 +0000 UTC m=+1540.164320677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-maas-api-tls") pod "maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx" (UID: "21e5382c-ab31-4fd4-a28d-e0823c47220d") : secret "maas-api-e2e-ipp-a-c0ec41-serving-cert" not found
Sep 10 16:07:23.748902 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.748883    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e5382c-ab31-4fd4-a28d-e0823c47220d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.750780 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.750756    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-metrics-certs\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:23.761420 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:23.761389    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rdht\" (UniqueName: \"kubernetes.io/projected/21e5382c-ab31-4fd4-a28d-e0823c47220d-kube-api-access-9rdht\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:24.253865 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.253815    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:24.256718 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.256683    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:24.359459 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.359419    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"]
Sep 10 16:07:24.363422 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.363394    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:24.366731 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.366704    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c0ec41-dockercfg-fnc9z\""
Sep 10 16:07:24.366865 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.366705    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c0ec41\""
Sep 10 16:07:24.378063 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.378029    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"]
Sep 10 16:07:24.455895 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.455856    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8qc5\" (UniqueName: \"kubernetes.io/projected/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-kube-api-access-g8qc5\") pod \"payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2\" (UID: \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:24.456096 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.455973    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2\" (UID: \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:24.523665 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.523559    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:24.556947 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.556906    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2\" (UID: \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:24.557171 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.556997    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8qc5\" (UniqueName: \"kubernetes.io/projected/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-kube-api-access-g8qc5\") pod \"payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2\" (UID: \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:24.557990 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.557955    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2\" (UID: \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:24.566397 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.566360    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8qc5\" (UniqueName: \"kubernetes.io/projected/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-kube-api-access-g8qc5\") pod \"payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2\" (UID: \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:24.675415 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.675372    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:24.810196 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.809945    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"]
Sep 10 16:07:24.959419 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:24.959391    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"]
Sep 10 16:07:24.961784 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:07:24.961737    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9923e511_2ba4_4aae_9ec7_11ab3c8bfe78.slice/crio-c0de9ec5815f17fb058e8d71c63991b9a534759a5d2adc625cfa0a3b5486987a WatchSource:0}: Error finding container c0de9ec5815f17fb058e8d71c63991b9a534759a5d2adc625cfa0a3b5486987a: Status 404 returned error can't find the container with id c0de9ec5815f17fb058e8d71c63991b9a534759a5d2adc625cfa0a3b5486987a
Sep 10 16:07:25.001370 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:25.001332    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx" event={"ID":"21e5382c-ab31-4fd4-a28d-e0823c47220d","Type":"ContainerStarted","Data":"a755c973bbac41dc48ebd07163f8df7a721459c9f1871eebc2cdea4a447c23f8"}
Sep 10 16:07:25.002567 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:25.002544    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2" event={"ID":"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78","Type":"ContainerStarted","Data":"c0de9ec5815f17fb058e8d71c63991b9a534759a5d2adc625cfa0a3b5486987a"}
Sep 10 16:07:26.010442 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:26.010347    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2" event={"ID":"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78","Type":"ContainerStarted","Data":"98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e"}
Sep 10 16:07:26.010442 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:26.010424    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:26.015530 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:26.015490    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx" event={"ID":"21e5382c-ab31-4fd4-a28d-e0823c47220d","Type":"ContainerStarted","Data":"516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7"}
Sep 10 16:07:26.015780 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:26.015653    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:26.031609 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:26.031544    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2" podStartSLOduration=2.0315263630000002 podStartE2EDuration="2.031526363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:07:26.031185147 +0000 UTC m=+1541.947310046" watchObservedRunningTime="2026-09-10 16:07:26.031526363 +0000 UTC m=+1541.947651241"
Sep 10 16:07:26.058738 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:26.058665    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx" podStartSLOduration=2.322246825 podStartE2EDuration="3.058641548s" totalImagesPullingTime="736.394723ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:07:26.054928275 +0000 UTC m=+1541.971053190" watchObservedRunningTime="2026-09-10 16:07:26.058641548 +0000 UTC m=+1541.974766429"
Sep 10 16:07:28.000002 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:27.999966    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"]
Sep 10 16:07:28.000513 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.000215    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5" podUID="1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" containerName="maas-api" containerID="cri-o://9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e" gracePeriod=30
Sep 10 16:07:28.268961 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.268927    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:07:28.400445 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.400407    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-maas-api-tls\") pod \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") "
Sep 10 16:07:28.400445 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.400448    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-openshift-service-ca\") pod \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") "
Sep 10 16:07:28.400718 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.400512    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-metrics-certs\") pod \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") "
Sep 10 16:07:28.400718 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.400542    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c92k7\" (UniqueName: \"kubernetes.io/projected/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-kube-api-access-c92k7\") pod \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\" (UID: \"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6\") "
Sep 10 16:07:28.400903 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.400872    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" (UID: "1aa11e5e-a478-4d74-a6ad-e063f6b6efd6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:07:28.402747 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.402718    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" (UID: "1aa11e5e-a478-4d74-a6ad-e063f6b6efd6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:07:28.402891 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.402804    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" (UID: "1aa11e5e-a478-4d74-a6ad-e063f6b6efd6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:07:28.402891 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.402840    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-kube-api-access-c92k7" (OuterVolumeSpecName: "kube-api-access-c92k7") pod "1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" (UID: "1aa11e5e-a478-4d74-a6ad-e063f6b6efd6"). InnerVolumeSpecName "kube-api-access-c92k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:07:28.502087 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.501997    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c92k7\" (UniqueName: \"kubernetes.io/projected/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-kube-api-access-c92k7\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:28.502087 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.502027    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:28.502087 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.502037    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:28.502087 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:28.502046    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:29.038792 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:29.038752    2527 generic.go:356] "Generic (PLEG): container finished" podID="1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" containerID="9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e" exitCode=0
Sep 10 16:07:29.038792 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:29.038794    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"
Sep 10 16:07:29.039301 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:29.038826    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5" event={"ID":"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6","Type":"ContainerDied","Data":"9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e"}
Sep 10 16:07:29.039301 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:29.038882    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5" event={"ID":"1aa11e5e-a478-4d74-a6ad-e063f6b6efd6","Type":"ContainerDied","Data":"e2127313e9c48694c358fa6cd289e600eba6483a9b0c1fdb19c797b9f83ac4a1"}
Sep 10 16:07:29.039301 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:29.038905    2527 scope.go:117] "RemoveContainer" containerID="9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e"
Sep 10 16:07:29.049187 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:29.049160    2527 scope.go:117] "RemoveContainer" containerID="9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e"
Sep 10 16:07:29.049557 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:07:29.049538    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e\": container with ID starting with 9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e not found: ID does not exist" containerID="9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e"
Sep 10 16:07:29.049656 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:29.049568    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e"} err="failed to get container status \"9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e\": rpc error: code = NotFound desc = could not find container \"9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e\": container with ID starting with 9706c31b55c2fd614512e5daba5fd8756557f0605b98f58ed87fb70d3ee2523e not found: ID does not exist"
Sep 10 16:07:29.060587 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:29.060554    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"]
Sep 10 16:07:29.062570 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:29.062546    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-81d3ea-84554bdb95-kdzx5"]
Sep 10 16:07:30.564110 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:30.564071    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" path="/var/lib/kubelet/pods/1aa11e5e-a478-4d74-a6ad-e063f6b6efd6/volumes"
Sep 10 16:07:32.027104 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:32.027074    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:07:37.023944 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:37.023905    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:07:46.148269 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.148233    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"]
Sep 10 16:07:46.149015 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.148965    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz" podUID="3abbba77-b006-4d29-9c33-5426702cac07" containerName="maas-api" containerID="cri-o://06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a" gracePeriod=30
Sep 10 16:07:46.419078 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.419047    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:07:46.482377 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.482334    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqkjz\" (UniqueName: \"kubernetes.io/projected/3abbba77-b006-4d29-9c33-5426702cac07-kube-api-access-wqkjz\") pod \"3abbba77-b006-4d29-9c33-5426702cac07\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") "
Sep 10 16:07:46.482602 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.482418    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abbba77-b006-4d29-9c33-5426702cac07-openshift-service-ca\") pod \"3abbba77-b006-4d29-9c33-5426702cac07\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") "
Sep 10 16:07:46.482602 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.482456    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-metrics-certs\") pod \"3abbba77-b006-4d29-9c33-5426702cac07\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") "
Sep 10 16:07:46.482602 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.482483    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-maas-api-tls\") pod \"3abbba77-b006-4d29-9c33-5426702cac07\" (UID: \"3abbba77-b006-4d29-9c33-5426702cac07\") "
Sep 10 16:07:46.482842 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.482812    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3abbba77-b006-4d29-9c33-5426702cac07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3abbba77-b006-4d29-9c33-5426702cac07" (UID: "3abbba77-b006-4d29-9c33-5426702cac07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:07:46.484827 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.484789    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3abbba77-b006-4d29-9c33-5426702cac07-kube-api-access-wqkjz" (OuterVolumeSpecName: "kube-api-access-wqkjz") pod "3abbba77-b006-4d29-9c33-5426702cac07" (UID: "3abbba77-b006-4d29-9c33-5426702cac07"). InnerVolumeSpecName "kube-api-access-wqkjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:07:46.484961 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.484833    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3abbba77-b006-4d29-9c33-5426702cac07" (UID: "3abbba77-b006-4d29-9c33-5426702cac07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:07:46.484961 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.484856    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3abbba77-b006-4d29-9c33-5426702cac07" (UID: "3abbba77-b006-4d29-9c33-5426702cac07"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:07:46.583523 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.583494    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abbba77-b006-4d29-9c33-5426702cac07-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:46.583523 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.583520    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:46.583523 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.583530    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abbba77-b006-4d29-9c33-5426702cac07-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:46.583749 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:46.583539    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqkjz\" (UniqueName: \"kubernetes.io/projected/3abbba77-b006-4d29-9c33-5426702cac07-kube-api-access-wqkjz\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:07:47.142526 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:47.142487    2527 generic.go:356] "Generic (PLEG): container finished" podID="3abbba77-b006-4d29-9c33-5426702cac07" containerID="06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a" exitCode=0
Sep 10 16:07:47.142687 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:47.142531    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"
Sep 10 16:07:47.142687 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:47.142565    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz" event={"ID":"3abbba77-b006-4d29-9c33-5426702cac07","Type":"ContainerDied","Data":"06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a"}
Sep 10 16:07:47.142687 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:47.142612    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz" event={"ID":"3abbba77-b006-4d29-9c33-5426702cac07","Type":"ContainerDied","Data":"6164cca8eec41a9dd6dba5ac22c3d79018fe382292120f9617ea4503d679a8cd"}
Sep 10 16:07:47.142687 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:47.142638    2527 scope.go:117] "RemoveContainer" containerID="06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a"
Sep 10 16:07:47.152293 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:47.152064    2527 scope.go:117] "RemoveContainer" containerID="06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a"
Sep 10 16:07:47.152537 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:07:47.152414    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a\": container with ID starting with 06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a not found: ID does not exist" containerID="06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a"
Sep 10 16:07:47.152537 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:47.152443    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a"} err="failed to get container status \"06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a\": rpc error: code = NotFound desc = could not find container \"06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a\": container with ID starting with 06dd86a0fc9f80b62bd423da3c41b5fa3b03700886cae09ce652cad42060dc4a not found: ID does not exist"
Sep 10 16:07:47.161303 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:47.161271    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"]
Sep 10 16:07:47.167317 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:47.167287    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a361a9-784974cb54-5bltz"]
Sep 10 16:07:48.571051 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:48.571010    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3abbba77-b006-4d29-9c33-5426702cac07" path="/var/lib/kubelet/pods/3abbba77-b006-4d29-9c33-5426702cac07/volumes"
Sep 10 16:07:52.438605 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.438565    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"]
Sep 10 16:07:52.439286 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.439267    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" containerName="maas-api"
Sep 10 16:07:52.439339 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.439290    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" containerName="maas-api"
Sep 10 16:07:52.439339 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.439315    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3abbba77-b006-4d29-9c33-5426702cac07" containerName="maas-api"
Sep 10 16:07:52.439339 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.439324    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="3abbba77-b006-4d29-9c33-5426702cac07" containerName="maas-api"
Sep 10 16:07:52.439436 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.439430    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="3abbba77-b006-4d29-9c33-5426702cac07" containerName="maas-api"
Sep 10 16:07:52.439469 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.439442    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aa11e5e-a478-4d74-a6ad-e063f6b6efd6" containerName="maas-api"
Sep 10 16:07:52.444725 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.444690    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.448847 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.448820    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-e9f330-serving-cert\""
Sep 10 16:07:52.455051 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.454889    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"]
Sep 10 16:07:52.545607 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.545568    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.545806 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.545747    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-metrics-certs\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.545806 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.545780    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj24v\" (UniqueName: \"kubernetes.io/projected/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-kube-api-access-kj24v\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.545915 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.545808    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.647394 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.647346    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-metrics-certs\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.647628 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.647411    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj24v\" (UniqueName: \"kubernetes.io/projected/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-kube-api-access-kj24v\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.647628 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.647449    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.647628 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:07:52.647579    2527 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-serving-cert: secret "maas-api-e2e-ipp-b-e9f330-serving-cert" not found
Sep 10 16:07:52.647944 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:07:52.647905    2527 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-maas-api-tls podName:d6b14fbb-7835-4c6b-8d6c-d7b980d041ff nodeName:}" failed. No retries permitted until 2026-09-10 16:07:53.147877423 +0000 UTC m=+1569.064002283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-maas-api-tls") pod "maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65" (UID: "d6b14fbb-7835-4c6b-8d6c-d7b980d041ff") : secret "maas-api-e2e-ipp-b-e9f330-serving-cert" not found
Sep 10 16:07:52.648196 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.648168    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.648868 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.648846    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.650562 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.650522    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-metrics-certs\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:52.659126 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:52.659102    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj24v\" (UniqueName: \"kubernetes.io/projected/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-kube-api-access-kj24v\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:53.153196 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.153155    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:53.155633 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.155607    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:53.225450 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.225411    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"]
Sep 10 16:07:53.229498 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.229472    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:07:53.232076 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.232053    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e9f330-dockercfg-96n5d\""
Sep 10 16:07:53.232231 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.232053    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e9f330\""
Sep 10 16:07:53.245967 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.245941    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"]
Sep 10 16:07:53.253719 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.253686    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438518e2-d5b4-451a-a108-1eb8ac389dc6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj\" (UID: \"438518e2-d5b4-451a-a108-1eb8ac389dc6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:07:53.253874 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.253741    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p99sk\" (UniqueName: \"kubernetes.io/projected/438518e2-d5b4-451a-a108-1eb8ac389dc6-kube-api-access-p99sk\") pod \"payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj\" (UID: \"438518e2-d5b4-451a-a108-1eb8ac389dc6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:07:53.354374 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.354340    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438518e2-d5b4-451a-a108-1eb8ac389dc6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj\" (UID: \"438518e2-d5b4-451a-a108-1eb8ac389dc6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:07:53.354596 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.354397    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p99sk\" (UniqueName: \"kubernetes.io/projected/438518e2-d5b4-451a-a108-1eb8ac389dc6-kube-api-access-p99sk\") pod \"payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj\" (UID: \"438518e2-d5b4-451a-a108-1eb8ac389dc6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:07:53.355341 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.355311    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438518e2-d5b4-451a-a108-1eb8ac389dc6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj\" (UID: \"438518e2-d5b4-451a-a108-1eb8ac389dc6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:07:53.360195 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.360168    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:53.366269 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.366244    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p99sk\" (UniqueName: \"kubernetes.io/projected/438518e2-d5b4-451a-a108-1eb8ac389dc6-kube-api-access-p99sk\") pod \"payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj\" (UID: \"438518e2-d5b4-451a-a108-1eb8ac389dc6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:07:53.541151 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.541079    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:07:53.648157 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.648098    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"]
Sep 10 16:07:53.649742 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:07:53.649707    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6b14fbb_7835_4c6b_8d6c_d7b980d041ff.slice/crio-8ab4518a474dff58f5a696976dbe43d044130dfcbb6d0822ab0f258d35e82ec2 WatchSource:0}: Error finding container 8ab4518a474dff58f5a696976dbe43d044130dfcbb6d0822ab0f258d35e82ec2: Status 404 returned error can't find the container with id 8ab4518a474dff58f5a696976dbe43d044130dfcbb6d0822ab0f258d35e82ec2
Sep 10 16:07:53.824058 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:53.824033    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"]
Sep 10 16:07:53.826346 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:07:53.826315    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod438518e2_d5b4_451a_a108_1eb8ac389dc6.slice/crio-9191b347cbc2b2909223ae5a61936ad5d141a3f4c4e0d9ef79a101903dcbde84 WatchSource:0}: Error finding container 9191b347cbc2b2909223ae5a61936ad5d141a3f4c4e0d9ef79a101903dcbde84: Status 404 returned error can't find the container with id 9191b347cbc2b2909223ae5a61936ad5d141a3f4c4e0d9ef79a101903dcbde84
Sep 10 16:07:54.184596 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:54.184553    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj" event={"ID":"438518e2-d5b4-451a-a108-1eb8ac389dc6","Type":"ContainerStarted","Data":"7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a"}
Sep 10 16:07:54.184883 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:54.184855    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj" event={"ID":"438518e2-d5b4-451a-a108-1eb8ac389dc6","Type":"ContainerStarted","Data":"9191b347cbc2b2909223ae5a61936ad5d141a3f4c4e0d9ef79a101903dcbde84"}
Sep 10 16:07:54.184989 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:54.184900    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:07:54.186239 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:54.186194    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65" event={"ID":"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff","Type":"ContainerStarted","Data":"8ab4518a474dff58f5a696976dbe43d044130dfcbb6d0822ab0f258d35e82ec2"}
Sep 10 16:07:54.207678 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:54.207627    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj" podStartSLOduration=1.20758813 podStartE2EDuration="1.20758813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:07:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:07:54.206039293 +0000 UTC m=+1570.122164177" watchObservedRunningTime="2026-09-10 16:07:54.20758813 +0000 UTC m=+1570.123713008"
Sep 10 16:07:55.195303 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:55.195264    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65" event={"ID":"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff","Type":"ContainerStarted","Data":"6255cb47f9a46ebe3d0ab2302d8028810d4e377ba2e83f4ed46eb25d8d624c63"}
Sep 10 16:07:55.195873 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:55.195572    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:07:55.216539 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:07:55.216474    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65" podStartSLOduration=2.749155864 podStartE2EDuration="3.216454217s" totalImagesPullingTime="467.298353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:07:55.214337195 +0000 UTC m=+1571.130462065" watchObservedRunningTime="2026-09-10 16:07:55.216454217 +0000 UTC m=+1571.132579095"
Sep 10 16:08:01.206756 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:01.206721    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:08:05.198684 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:05.198654    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:08:05.942624 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:05.942585    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"]
Sep 10 16:08:05.942881 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:05.942854    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c" podUID="47b3ab5b-def6-41f1-a7a7-2494419781ee" containerName="maas-api" containerID="cri-o://40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0" gracePeriod=30
Sep 10 16:08:06.220074 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.220042    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:08:06.257648 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.257613    2527 generic.go:356] "Generic (PLEG): container finished" podID="47b3ab5b-def6-41f1-a7a7-2494419781ee" containerID="40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0" exitCode=0
Sep 10 16:08:06.257817 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.257679    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"
Sep 10 16:08:06.257817 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.257695    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c" event={"ID":"47b3ab5b-def6-41f1-a7a7-2494419781ee","Type":"ContainerDied","Data":"40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0"}
Sep 10 16:08:06.257817 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.257756    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c" event={"ID":"47b3ab5b-def6-41f1-a7a7-2494419781ee","Type":"ContainerDied","Data":"2511565094aecc80e119111432e9b18a6269c2fb79fd7781d606cacd1538bc3e"}
Sep 10 16:08:06.257817 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.257779    2527 scope.go:117] "RemoveContainer" containerID="40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0"
Sep 10 16:08:06.267362 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.267336    2527 scope.go:117] "RemoveContainer" containerID="40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0"
Sep 10 16:08:06.267743 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:08:06.267723    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0\": container with ID starting with 40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0 not found: ID does not exist" containerID="40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0"
Sep 10 16:08:06.267809 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.267756    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0"} err="failed to get container status \"40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0\": rpc error: code = NotFound desc = could not find container \"40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0\": container with ID starting with 40470fd0d158f448c815087ca347a677b0f3bd0c851dcf2f70442e00eabfccf0 not found: ID does not exist"
Sep 10 16:08:06.282208 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.282166    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-maas-api-tls\") pod \"47b3ab5b-def6-41f1-a7a7-2494419781ee\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") "
Sep 10 16:08:06.282381 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.282287    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b3ab5b-def6-41f1-a7a7-2494419781ee-openshift-service-ca\") pod \"47b3ab5b-def6-41f1-a7a7-2494419781ee\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") "
Sep 10 16:08:06.282381 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.282352    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-metrics-certs\") pod \"47b3ab5b-def6-41f1-a7a7-2494419781ee\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") "
Sep 10 16:08:06.282494 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.282401    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8gzz\" (UniqueName: \"kubernetes.io/projected/47b3ab5b-def6-41f1-a7a7-2494419781ee-kube-api-access-q8gzz\") pod \"47b3ab5b-def6-41f1-a7a7-2494419781ee\" (UID: \"47b3ab5b-def6-41f1-a7a7-2494419781ee\") "
Sep 10 16:08:06.282663 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.282636    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47b3ab5b-def6-41f1-a7a7-2494419781ee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "47b3ab5b-def6-41f1-a7a7-2494419781ee" (UID: "47b3ab5b-def6-41f1-a7a7-2494419781ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:08:06.284546 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.284508    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "47b3ab5b-def6-41f1-a7a7-2494419781ee" (UID: "47b3ab5b-def6-41f1-a7a7-2494419781ee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:08:06.284752 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.284576    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "47b3ab5b-def6-41f1-a7a7-2494419781ee" (UID: "47b3ab5b-def6-41f1-a7a7-2494419781ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:08:06.284752 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.284594    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b3ab5b-def6-41f1-a7a7-2494419781ee-kube-api-access-q8gzz" (OuterVolumeSpecName: "kube-api-access-q8gzz") pod "47b3ab5b-def6-41f1-a7a7-2494419781ee" (UID: "47b3ab5b-def6-41f1-a7a7-2494419781ee"). InnerVolumeSpecName "kube-api-access-q8gzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:08:06.383732 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.383688    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8gzz\" (UniqueName: \"kubernetes.io/projected/47b3ab5b-def6-41f1-a7a7-2494419781ee-kube-api-access-q8gzz\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:08:06.383732 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.383721    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:08:06.383732 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.383732    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47b3ab5b-def6-41f1-a7a7-2494419781ee-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:08:06.383732 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.383742    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47b3ab5b-def6-41f1-a7a7-2494419781ee-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:08:06.584776 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.584741    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"]
Sep 10 16:08:06.591169 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:06.591127    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-33451c-5d87bd8695-sl84c"]
Sep 10 16:08:08.564787 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:08.564747    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47b3ab5b-def6-41f1-a7a7-2494419781ee" path="/var/lib/kubelet/pods/47b3ab5b-def6-41f1-a7a7-2494419781ee/volumes"
Sep 10 16:08:21.183373 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:21.183326    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-88prc_e8cb2ee3-5670-4eb0-b31d-afa362698dd5/payload-processing/0.log"
Sep 10 16:08:21.306178 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:21.306148    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj_438518e2-d5b4-451a-a108-1eb8ac389dc6/payload-processing/0.log"
Sep 10 16:08:57.072645 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:57.072603    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-88prc_e8cb2ee3-5670-4eb0-b31d-afa362698dd5/payload-processing/0.log"
Sep 10 16:08:57.199733 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:08:57.199702    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj_438518e2-d5b4-451a-a108-1eb8ac389dc6/payload-processing/0.log"
Sep 10 16:09:59.180267 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.180233    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"]
Sep 10 16:09:59.180769 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.180450    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx" podUID="21e5382c-ab31-4fd4-a28d-e0823c47220d" containerName="maas-api" containerID="cri-o://516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7" gracePeriod=30
Sep 10 16:09:59.263798 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.263754    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"]
Sep 10 16:09:59.264046 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.264019    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2" podUID="9923e511-2ba4-4aae-9ec7-11ab3c8bfe78" containerName="payload-pre-processing" containerID="cri-o://98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e" gracePeriod=30
Sep 10 16:09:59.530933 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.530902    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:09:59.574122 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.574085    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:09:59.594245 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.594209    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rdht\" (UniqueName: \"kubernetes.io/projected/21e5382c-ab31-4fd4-a28d-e0823c47220d-kube-api-access-9rdht\") pod \"21e5382c-ab31-4fd4-a28d-e0823c47220d\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") "
Sep 10 16:09:59.594455 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.594264    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e5382c-ab31-4fd4-a28d-e0823c47220d-openshift-service-ca\") pod \"21e5382c-ab31-4fd4-a28d-e0823c47220d\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") "
Sep 10 16:09:59.594455 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.594298    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-metrics-certs\") pod \"21e5382c-ab31-4fd4-a28d-e0823c47220d\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") "
Sep 10 16:09:59.594455 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.594366    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-maas-api-tls\") pod \"21e5382c-ab31-4fd4-a28d-e0823c47220d\" (UID: \"21e5382c-ab31-4fd4-a28d-e0823c47220d\") "
Sep 10 16:09:59.594765 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.594710    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21e5382c-ab31-4fd4-a28d-e0823c47220d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "21e5382c-ab31-4fd4-a28d-e0823c47220d" (UID: "21e5382c-ab31-4fd4-a28d-e0823c47220d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:09:59.595023 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.594990    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e5382c-ab31-4fd4-a28d-e0823c47220d-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:09:59.596681 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.596652    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "21e5382c-ab31-4fd4-a28d-e0823c47220d" (UID: "21e5382c-ab31-4fd4-a28d-e0823c47220d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:09:59.597047 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.597022    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "21e5382c-ab31-4fd4-a28d-e0823c47220d" (UID: "21e5382c-ab31-4fd4-a28d-e0823c47220d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:09:59.597157 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.597059    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21e5382c-ab31-4fd4-a28d-e0823c47220d-kube-api-access-9rdht" (OuterVolumeSpecName: "kube-api-access-9rdht") pod "21e5382c-ab31-4fd4-a28d-e0823c47220d" (UID: "21e5382c-ab31-4fd4-a28d-e0823c47220d"). InnerVolumeSpecName "kube-api-access-9rdht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:09:59.696398 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.696280    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8qc5\" (UniqueName: \"kubernetes.io/projected/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-kube-api-access-g8qc5\") pod \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\" (UID: \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\") "
Sep 10 16:09:59.696398 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.696373    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-plugins-config-volume\") pod \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\" (UID: \"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78\") "
Sep 10 16:09:59.696662 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.696556    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:09:59.696662 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.696567    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21e5382c-ab31-4fd4-a28d-e0823c47220d-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:09:59.696662 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.696577    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rdht\" (UniqueName: \"kubernetes.io/projected/21e5382c-ab31-4fd4-a28d-e0823c47220d-kube-api-access-9rdht\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:09:59.696808 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.696746    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9923e511-2ba4-4aae-9ec7-11ab3c8bfe78" (UID: "9923e511-2ba4-4aae-9ec7-11ab3c8bfe78"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:09:59.698598 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.698570    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-kube-api-access-g8qc5" (OuterVolumeSpecName: "kube-api-access-g8qc5") pod "9923e511-2ba4-4aae-9ec7-11ab3c8bfe78" (UID: "9923e511-2ba4-4aae-9ec7-11ab3c8bfe78"). InnerVolumeSpecName "kube-api-access-g8qc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:09:59.797398 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.797336    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8qc5\" (UniqueName: \"kubernetes.io/projected/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-kube-api-access-g8qc5\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:09:59.797398 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.797388    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:09:59.894125 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.894084    2527 generic.go:356] "Generic (PLEG): container finished" podID="9923e511-2ba4-4aae-9ec7-11ab3c8bfe78" containerID="98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e" exitCode=0
Sep 10 16:09:59.894350 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.894146    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"
Sep 10 16:09:59.894350 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.894166    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2" event={"ID":"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78","Type":"ContainerDied","Data":"98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e"}
Sep 10 16:09:59.894350 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.894245    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2" event={"ID":"9923e511-2ba4-4aae-9ec7-11ab3c8bfe78","Type":"ContainerDied","Data":"c0de9ec5815f17fb058e8d71c63991b9a534759a5d2adc625cfa0a3b5486987a"}
Sep 10 16:09:59.894350 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.894271    2527 scope.go:117] "RemoveContainer" containerID="98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e"
Sep 10 16:09:59.895907 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.895879    2527 generic.go:356] "Generic (PLEG): container finished" podID="21e5382c-ab31-4fd4-a28d-e0823c47220d" containerID="516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7" exitCode=0
Sep 10 16:09:59.896055 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.895945    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"
Sep 10 16:09:59.896055 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.895949    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx" event={"ID":"21e5382c-ab31-4fd4-a28d-e0823c47220d","Type":"ContainerDied","Data":"516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7"}
Sep 10 16:09:59.896215 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.896059    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx" event={"ID":"21e5382c-ab31-4fd4-a28d-e0823c47220d","Type":"ContainerDied","Data":"a755c973bbac41dc48ebd07163f8df7a721459c9f1871eebc2cdea4a447c23f8"}
Sep 10 16:09:59.904868 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.904839    2527 scope.go:117] "RemoveContainer" containerID="98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e"
Sep 10 16:09:59.905277 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:09:59.905257    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e\": container with ID starting with 98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e not found: ID does not exist" containerID="98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e"
Sep 10 16:09:59.905369 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.905285    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e"} err="failed to get container status \"98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e\": rpc error: code = NotFound desc = could not find container \"98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e\": container with ID starting with 98902a473653ace3390ffe74c7ede423a40f01eb6252cd92aabcb582dafbae9e not found: ID does not exist"
Sep 10 16:09:59.905369 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.905302    2527 scope.go:117] "RemoveContainer" containerID="516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7"
Sep 10 16:09:59.914392 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.914370    2527 scope.go:117] "RemoveContainer" containerID="516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7"
Sep 10 16:09:59.914720 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:09:59.914698    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7\": container with ID starting with 516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7 not found: ID does not exist" containerID="516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7"
Sep 10 16:09:59.914795 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.914731    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7"} err="failed to get container status \"516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7\": rpc error: code = NotFound desc = could not find container \"516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7\": container with ID starting with 516845c1346845c0c91ac51ce11d4e2e28fdf9d9fe3b4bef5e9f74f7a0fecbc7 not found: ID does not exist"
Sep 10 16:09:59.921260 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.921234    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"]
Sep 10 16:09:59.925526 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.925498    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c0ec41-865d7d9554-94wp2"]
Sep 10 16:09:59.940824 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.940785    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"]
Sep 10 16:09:59.950656 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:09:59.950561    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c0ec41-74d457688f-f9khx"]
Sep 10 16:10:00.565457 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:10:00.565419    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21e5382c-ab31-4fd4-a28d-e0823c47220d" path="/var/lib/kubelet/pods/21e5382c-ab31-4fd4-a28d-e0823c47220d/volumes"
Sep 10 16:10:00.565855 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:10:00.565822    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9923e511-2ba4-4aae-9ec7-11ab3c8bfe78" path="/var/lib/kubelet/pods/9923e511-2ba4-4aae-9ec7-11ab3c8bfe78/volumes"
Sep 10 16:11:01.102518 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.102468    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"]
Sep 10 16:11:01.103012 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.102730    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65" podUID="d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" containerName="maas-api" containerID="cri-o://6255cb47f9a46ebe3d0ab2302d8028810d4e377ba2e83f4ed46eb25d8d624c63" gracePeriod=30
Sep 10 16:11:01.152011 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.151970    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"]
Sep 10 16:11:01.152270 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.152239    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj" podUID="438518e2-d5b4-451a-a108-1eb8ac389dc6" containerName="payload-processing" containerID="cri-o://7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a" gracePeriod=30
Sep 10 16:11:01.201621 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.201577    2527 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65" podUID="d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.72:8443/health\": dial tcp 10.132.0.72:8443: connect: connection refused"
Sep 10 16:11:01.232562 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.232522    2527 generic.go:356] "Generic (PLEG): container finished" podID="d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" containerID="6255cb47f9a46ebe3d0ab2302d8028810d4e377ba2e83f4ed46eb25d8d624c63" exitCode=0
Sep 10 16:11:01.232737 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.232595    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65" event={"ID":"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff","Type":"ContainerDied","Data":"6255cb47f9a46ebe3d0ab2302d8028810d4e377ba2e83f4ed46eb25d8d624c63"}
Sep 10 16:11:01.416305 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.416278    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:11:01.451360 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.451329    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:11:01.521972 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.521929    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-metrics-certs\") pod \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") "
Sep 10 16:11:01.522162 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.522043    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-maas-api-tls\") pod \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") "
Sep 10 16:11:01.522162 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.522087    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj24v\" (UniqueName: \"kubernetes.io/projected/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-kube-api-access-kj24v\") pod \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") "
Sep 10 16:11:01.522162 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.522114    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-openshift-service-ca\") pod \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\" (UID: \"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff\") "
Sep 10 16:11:01.522569 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.522534    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" (UID: "d6b14fbb-7835-4c6b-8d6c-d7b980d041ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:11:01.524241 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.524215    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" (UID: "d6b14fbb-7835-4c6b-8d6c-d7b980d041ff"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:11:01.524331 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.524256    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" (UID: "d6b14fbb-7835-4c6b-8d6c-d7b980d041ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:11:01.524331 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.524263    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-kube-api-access-kj24v" (OuterVolumeSpecName: "kube-api-access-kj24v") pod "d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" (UID: "d6b14fbb-7835-4c6b-8d6c-d7b980d041ff"). InnerVolumeSpecName "kube-api-access-kj24v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:11:01.622703 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.622604    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438518e2-d5b4-451a-a108-1eb8ac389dc6-plugins-config-volume\") pod \"438518e2-d5b4-451a-a108-1eb8ac389dc6\" (UID: \"438518e2-d5b4-451a-a108-1eb8ac389dc6\") "
Sep 10 16:11:01.622703 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.622646    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p99sk\" (UniqueName: \"kubernetes.io/projected/438518e2-d5b4-451a-a108-1eb8ac389dc6-kube-api-access-p99sk\") pod \"438518e2-d5b4-451a-a108-1eb8ac389dc6\" (UID: \"438518e2-d5b4-451a-a108-1eb8ac389dc6\") "
Sep 10 16:11:01.622938 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.622920    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:01.622938 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.622935    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj24v\" (UniqueName: \"kubernetes.io/projected/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-kube-api-access-kj24v\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:01.623024 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.622945    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:01.623024 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.622954    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:01.623024 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.622993    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/438518e2-d5b4-451a-a108-1eb8ac389dc6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "438518e2-d5b4-451a-a108-1eb8ac389dc6" (UID: "438518e2-d5b4-451a-a108-1eb8ac389dc6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:11:01.624819 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.624793    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/438518e2-d5b4-451a-a108-1eb8ac389dc6-kube-api-access-p99sk" (OuterVolumeSpecName: "kube-api-access-p99sk") pod "438518e2-d5b4-451a-a108-1eb8ac389dc6" (UID: "438518e2-d5b4-451a-a108-1eb8ac389dc6"). InnerVolumeSpecName "kube-api-access-p99sk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:11:01.723943 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.723894    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/438518e2-d5b4-451a-a108-1eb8ac389dc6-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:01.724189 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:01.724033    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p99sk\" (UniqueName: \"kubernetes.io/projected/438518e2-d5b4-451a-a108-1eb8ac389dc6-kube-api-access-p99sk\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:02.240688 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.240652    2527 generic.go:356] "Generic (PLEG): container finished" podID="438518e2-d5b4-451a-a108-1eb8ac389dc6" containerID="7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a" exitCode=0
Sep 10 16:11:02.241212 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.240697    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"
Sep 10 16:11:02.241212 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.240733    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj" event={"ID":"438518e2-d5b4-451a-a108-1eb8ac389dc6","Type":"ContainerDied","Data":"7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a"}
Sep 10 16:11:02.241212 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.240794    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj" event={"ID":"438518e2-d5b4-451a-a108-1eb8ac389dc6","Type":"ContainerDied","Data":"9191b347cbc2b2909223ae5a61936ad5d141a3f4c4e0d9ef79a101903dcbde84"}
Sep 10 16:11:02.241212 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.240819    2527 scope.go:117] "RemoveContainer" containerID="7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a"
Sep 10 16:11:02.242461 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.242436    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65" event={"ID":"d6b14fbb-7835-4c6b-8d6c-d7b980d041ff","Type":"ContainerDied","Data":"8ab4518a474dff58f5a696976dbe43d044130dfcbb6d0822ab0f258d35e82ec2"}
Sep 10 16:11:02.242583 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.242452    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"
Sep 10 16:11:02.250690 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.250663    2527 scope.go:117] "RemoveContainer" containerID="7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a"
Sep 10 16:11:02.251100 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:11:02.251062    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a\": container with ID starting with 7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a not found: ID does not exist" containerID="7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a"
Sep 10 16:11:02.251100 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.251090    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a"} err="failed to get container status \"7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a\": rpc error: code = NotFound desc = could not find container \"7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a\": container with ID starting with 7356fd95e047ba5f3ecaf6540a655c95b0885098f0c8b9b15c056d7bbe46196a not found: ID does not exist"
Sep 10 16:11:02.251244 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.251108    2527 scope.go:117] "RemoveContainer" containerID="6255cb47f9a46ebe3d0ab2302d8028810d4e377ba2e83f4ed46eb25d8d624c63"
Sep 10 16:11:02.269917 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.269885    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"]
Sep 10 16:11:02.275215 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.275189    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e9f330-89989bf75-wfhgj"]
Sep 10 16:11:02.285671 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.285640    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"]
Sep 10 16:11:02.292263 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.292238    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e9f330-5b785d4746-prx65"]
Sep 10 16:11:02.564761 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.564725    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="438518e2-d5b4-451a-a108-1eb8ac389dc6" path="/var/lib/kubelet/pods/438518e2-d5b4-451a-a108-1eb8ac389dc6/volumes"
Sep 10 16:11:02.565195 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:02.565179    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" path="/var/lib/kubelet/pods/d6b14fbb-7835-4c6b-8d6c-d7b980d041ff/volumes"
Sep 10 16:11:18.904956 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:18.904920    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"]
Sep 10 16:11:18.905532 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:18.905202    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6" podUID="018cae23-2e24-4116-a13b-a9db57cdcd8e" containerName="maas-api" containerID="cri-o://edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc" gracePeriod=30
Sep 10 16:11:19.164022 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.163992    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:11:19.192404 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.192369    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018cae23-2e24-4116-a13b-a9db57cdcd8e-openshift-service-ca\") pod \"018cae23-2e24-4116-a13b-a9db57cdcd8e\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") "
Sep 10 16:11:19.192620 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.192425    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clr9r\" (UniqueName: \"kubernetes.io/projected/018cae23-2e24-4116-a13b-a9db57cdcd8e-kube-api-access-clr9r\") pod \"018cae23-2e24-4116-a13b-a9db57cdcd8e\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") "
Sep 10 16:11:19.192620 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.192482    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-maas-api-tls\") pod \"018cae23-2e24-4116-a13b-a9db57cdcd8e\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") "
Sep 10 16:11:19.192620 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.192516    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-metrics-certs\") pod \"018cae23-2e24-4116-a13b-a9db57cdcd8e\" (UID: \"018cae23-2e24-4116-a13b-a9db57cdcd8e\") "
Sep 10 16:11:19.192872 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.192822    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/018cae23-2e24-4116-a13b-a9db57cdcd8e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "018cae23-2e24-4116-a13b-a9db57cdcd8e" (UID: "018cae23-2e24-4116-a13b-a9db57cdcd8e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:11:19.195811 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.195735    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "018cae23-2e24-4116-a13b-a9db57cdcd8e" (UID: "018cae23-2e24-4116-a13b-a9db57cdcd8e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:11:19.195965 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.195818    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "018cae23-2e24-4116-a13b-a9db57cdcd8e" (UID: "018cae23-2e24-4116-a13b-a9db57cdcd8e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:11:19.195965 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.195910    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/018cae23-2e24-4116-a13b-a9db57cdcd8e-kube-api-access-clr9r" (OuterVolumeSpecName: "kube-api-access-clr9r") pod "018cae23-2e24-4116-a13b-a9db57cdcd8e" (UID: "018cae23-2e24-4116-a13b-a9db57cdcd8e"). InnerVolumeSpecName "kube-api-access-clr9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:11:19.293881 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.293845    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clr9r\" (UniqueName: \"kubernetes.io/projected/018cae23-2e24-4116-a13b-a9db57cdcd8e-kube-api-access-clr9r\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:19.293881 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.293874    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:19.293881 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.293885    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/018cae23-2e24-4116-a13b-a9db57cdcd8e-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:19.294125 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.293894    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018cae23-2e24-4116-a13b-a9db57cdcd8e-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:19.336354 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.336318    2527 generic.go:356] "Generic (PLEG): container finished" podID="018cae23-2e24-4116-a13b-a9db57cdcd8e" containerID="edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc" exitCode=0
Sep 10 16:11:19.336543 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.336365    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"
Sep 10 16:11:19.336543 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.336411    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6" event={"ID":"018cae23-2e24-4116-a13b-a9db57cdcd8e","Type":"ContainerDied","Data":"edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc"}
Sep 10 16:11:19.336543 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.336485    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6" event={"ID":"018cae23-2e24-4116-a13b-a9db57cdcd8e","Type":"ContainerDied","Data":"2bfc0f1a6c38a8e4e721925cbd19b61ce326321a1fc50eb58f4597cdcd69b84e"}
Sep 10 16:11:19.336543 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.336519    2527 scope.go:117] "RemoveContainer" containerID="edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc"
Sep 10 16:11:19.347188 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.347169    2527 scope.go:117] "RemoveContainer" containerID="edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc"
Sep 10 16:11:19.347543 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:11:19.347524    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc\": container with ID starting with edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc not found: ID does not exist" containerID="edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc"
Sep 10 16:11:19.347594 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.347554    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc"} err="failed to get container status \"edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc\": rpc error: code = NotFound desc = could not find container \"edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc\": container with ID starting with edc990ad2a971341bb8148bc3ca6b18f7dbdf30a109899989aede3dd12ec71fc not found: ID does not exist"
Sep 10 16:11:19.360427 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.360393    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"]
Sep 10 16:11:19.366482 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:19.366439    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-50ed73-6985c64b5d-cf5m6"]
Sep 10 16:11:20.564424 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:20.564389    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="018cae23-2e24-4116-a13b-a9db57cdcd8e" path="/var/lib/kubelet/pods/018cae23-2e24-4116-a13b-a9db57cdcd8e/volumes"
Sep 10 16:11:37.225236 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.225107    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"]
Sep 10 16:11:37.225732 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.225385    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb" podUID="5956aa90-2b25-4fef-af35-2bf5486f6ef8" containerName="maas-api" containerID="cri-o://5e271eae6401524d1ce9265f5052bc0c6b85e8db4672d7c6231c05bc301205a4" gracePeriod=30
Sep 10 16:11:37.320812 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.320767    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"]
Sep 10 16:11:37.321635 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.321076    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8" podUID="3deb3119-081a-4839-a4ef-f0f0c4868d44" containerName="payload-pre-processing" containerID="cri-o://db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6" gracePeriod=30
Sep 10 16:11:37.437553 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.437508    2527 generic.go:356] "Generic (PLEG): container finished" podID="5956aa90-2b25-4fef-af35-2bf5486f6ef8" containerID="5e271eae6401524d1ce9265f5052bc0c6b85e8db4672d7c6231c05bc301205a4" exitCode=0
Sep 10 16:11:37.437753 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.437585    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb" event={"ID":"5956aa90-2b25-4fef-af35-2bf5486f6ef8","Type":"ContainerDied","Data":"5e271eae6401524d1ce9265f5052bc0c6b85e8db4672d7c6231c05bc301205a4"}
Sep 10 16:11:37.495915 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.495885    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:11:37.577280 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.577246    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-maas-api-tls\") pod \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") "
Sep 10 16:11:37.577536 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.577309    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfnck\" (UniqueName: \"kubernetes.io/projected/5956aa90-2b25-4fef-af35-2bf5486f6ef8-kube-api-access-nfnck\") pod \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") "
Sep 10 16:11:37.577536 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.577357    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5956aa90-2b25-4fef-af35-2bf5486f6ef8-openshift-service-ca\") pod \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") "
Sep 10 16:11:37.577536 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.577391    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-metrics-certs\") pod \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\" (UID: \"5956aa90-2b25-4fef-af35-2bf5486f6ef8\") "
Sep 10 16:11:37.577763 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.577730    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5956aa90-2b25-4fef-af35-2bf5486f6ef8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5956aa90-2b25-4fef-af35-2bf5486f6ef8" (UID: "5956aa90-2b25-4fef-af35-2bf5486f6ef8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:11:37.579615 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.579588    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5956aa90-2b25-4fef-af35-2bf5486f6ef8" (UID: "5956aa90-2b25-4fef-af35-2bf5486f6ef8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:11:37.579727 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.579680    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5956aa90-2b25-4fef-af35-2bf5486f6ef8-kube-api-access-nfnck" (OuterVolumeSpecName: "kube-api-access-nfnck") pod "5956aa90-2b25-4fef-af35-2bf5486f6ef8" (UID: "5956aa90-2b25-4fef-af35-2bf5486f6ef8"). InnerVolumeSpecName "kube-api-access-nfnck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:11:37.579812 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.579786    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5956aa90-2b25-4fef-af35-2bf5486f6ef8" (UID: "5956aa90-2b25-4fef-af35-2bf5486f6ef8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:11:37.605656 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.605611    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:11:37.679119 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.679074    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:37.679119 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.679108    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfnck\" (UniqueName: \"kubernetes.io/projected/5956aa90-2b25-4fef-af35-2bf5486f6ef8-kube-api-access-nfnck\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:37.679119 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.679119    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5956aa90-2b25-4fef-af35-2bf5486f6ef8-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:37.679488 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.679128    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5956aa90-2b25-4fef-af35-2bf5486f6ef8-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:37.780443 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.780410    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deb3119-081a-4839-a4ef-f0f0c4868d44-plugins-config-volume\") pod \"3deb3119-081a-4839-a4ef-f0f0c4868d44\" (UID: \"3deb3119-081a-4839-a4ef-f0f0c4868d44\") "
Sep 10 16:11:37.780612 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.780467    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwdbx\" (UniqueName: \"kubernetes.io/projected/3deb3119-081a-4839-a4ef-f0f0c4868d44-kube-api-access-cwdbx\") pod \"3deb3119-081a-4839-a4ef-f0f0c4868d44\" (UID: \"3deb3119-081a-4839-a4ef-f0f0c4868d44\") "
Sep 10 16:11:37.780893 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.780866    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3deb3119-081a-4839-a4ef-f0f0c4868d44-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3deb3119-081a-4839-a4ef-f0f0c4868d44" (UID: "3deb3119-081a-4839-a4ef-f0f0c4868d44"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:11:37.782677 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.782647    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3deb3119-081a-4839-a4ef-f0f0c4868d44-kube-api-access-cwdbx" (OuterVolumeSpecName: "kube-api-access-cwdbx") pod "3deb3119-081a-4839-a4ef-f0f0c4868d44" (UID: "3deb3119-081a-4839-a4ef-f0f0c4868d44"). InnerVolumeSpecName "kube-api-access-cwdbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:11:37.881915 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.881866    2527 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deb3119-081a-4839-a4ef-f0f0c4868d44-plugins-config-volume\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:37.881915 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:37.881904    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwdbx\" (UniqueName: \"kubernetes.io/projected/3deb3119-081a-4839-a4ef-f0f0c4868d44-kube-api-access-cwdbx\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:38.444565 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.444533    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"
Sep 10 16:11:38.445063 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.444523    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb" event={"ID":"5956aa90-2b25-4fef-af35-2bf5486f6ef8","Type":"ContainerDied","Data":"6644922840f1536edaac23078563ed1cab908cdff32d87de324dca42c2e34bed"}
Sep 10 16:11:38.445063 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.444698    2527 scope.go:117] "RemoveContainer" containerID="5e271eae6401524d1ce9265f5052bc0c6b85e8db4672d7c6231c05bc301205a4"
Sep 10 16:11:38.446234 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.446208    2527 generic.go:356] "Generic (PLEG): container finished" podID="3deb3119-081a-4839-a4ef-f0f0c4868d44" containerID="db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6" exitCode=0
Sep 10 16:11:38.446317 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.446255    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"
Sep 10 16:11:38.446317 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.446291    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8" event={"ID":"3deb3119-081a-4839-a4ef-f0f0c4868d44","Type":"ContainerDied","Data":"db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6"}
Sep 10 16:11:38.446403 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.446343    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8" event={"ID":"3deb3119-081a-4839-a4ef-f0f0c4868d44","Type":"ContainerDied","Data":"9fcabaff0b7a3424ab3c8e7a47259182d3ba895bdd9671b91a07d596a3267d31"}
Sep 10 16:11:38.454326 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.454300    2527 scope.go:117] "RemoveContainer" containerID="db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6"
Sep 10 16:11:38.463383 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.463361    2527 scope.go:117] "RemoveContainer" containerID="db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6"
Sep 10 16:11:38.463693 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:11:38.463670    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6\": container with ID starting with db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6 not found: ID does not exist" containerID="db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6"
Sep 10 16:11:38.463741 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.463704    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6"} err="failed to get container status \"db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6\": rpc error: code = NotFound desc = could not find container \"db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6\": container with ID starting with db8e1ee33663e9a321cde5e1c5f13c6459fead015f3772ea079f0c51f87640d6 not found: ID does not exist"
Sep 10 16:11:38.478511 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.478473    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"]
Sep 10 16:11:38.484241 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.484206    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-67df2b-65cdd5d86d-ltcpb"]
Sep 10 16:11:38.494152 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.494050    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"]
Sep 10 16:11:38.498240 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.498213    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67df2b-5cb55679d6-lplm8"]
Sep 10 16:11:38.564302 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.564267    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3deb3119-081a-4839-a4ef-f0f0c4868d44" path="/var/lib/kubelet/pods/3deb3119-081a-4839-a4ef-f0f0c4868d44/volumes"
Sep 10 16:11:38.564690 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:38.564675    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5956aa90-2b25-4fef-af35-2bf5486f6ef8" path="/var/lib/kubelet/pods/5956aa90-2b25-4fef-af35-2bf5486f6ef8/volumes"
Sep 10 16:11:44.667781 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:44.667748    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 16:11:44.675098 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:44.675071    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 16:11:48.554807 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.554764    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-vp468"]
Sep 10 16:11:48.555272 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.554980    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-69fdcbc57-vp468" podUID="d79009a5-1cda-4084-98c0-c9c90fdbc564" containerName="manager" containerID="cri-o://aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d" gracePeriod=30
Sep 10 16:11:48.819956 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.819882    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 16:11:48.888035 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.888001    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-webhook-cert\") pod \"d79009a5-1cda-4084-98c0-c9c90fdbc564\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") "
Sep 10 16:11:48.888276 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.888055    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf7zs\" (UniqueName: \"kubernetes.io/projected/d79009a5-1cda-4084-98c0-c9c90fdbc564-kube-api-access-kf7zs\") pod \"d79009a5-1cda-4084-98c0-c9c90fdbc564\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") "
Sep 10 16:11:48.888276 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.888103    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-metrics-certs\") pod \"d79009a5-1cda-4084-98c0-c9c90fdbc564\" (UID: \"d79009a5-1cda-4084-98c0-c9c90fdbc564\") "
Sep 10 16:11:48.890328 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.890290    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d79009a5-1cda-4084-98c0-c9c90fdbc564" (UID: "d79009a5-1cda-4084-98c0-c9c90fdbc564"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:11:48.890328 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.890316    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d79009a5-1cda-4084-98c0-c9c90fdbc564" (UID: "d79009a5-1cda-4084-98c0-c9c90fdbc564"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:11:48.890510 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.890394    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d79009a5-1cda-4084-98c0-c9c90fdbc564-kube-api-access-kf7zs" (OuterVolumeSpecName: "kube-api-access-kf7zs") pod "d79009a5-1cda-4084-98c0-c9c90fdbc564" (UID: "d79009a5-1cda-4084-98c0-c9c90fdbc564"). InnerVolumeSpecName "kube-api-access-kf7zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:11:48.989407 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.989358    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:48.989407 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.989398    2527 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d79009a5-1cda-4084-98c0-c9c90fdbc564-webhook-cert\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:48.989407 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:48.989407    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf7zs\" (UniqueName: \"kubernetes.io/projected/d79009a5-1cda-4084-98c0-c9c90fdbc564-kube-api-access-kf7zs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:11:49.507860 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:49.507820    2527 generic.go:356] "Generic (PLEG): container finished" podID="d79009a5-1cda-4084-98c0-c9c90fdbc564" containerID="aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d" exitCode=0
Sep 10 16:11:49.508095 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:49.507874    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-vp468"
Sep 10 16:11:49.508095 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:49.507905    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-vp468" event={"ID":"d79009a5-1cda-4084-98c0-c9c90fdbc564","Type":"ContainerDied","Data":"aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d"}
Sep 10 16:11:49.508095 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:49.507962    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-vp468" event={"ID":"d79009a5-1cda-4084-98c0-c9c90fdbc564","Type":"ContainerDied","Data":"a72b8b4c008eb2911a4e8d5c06746a03a7d3100f912d72e21bcba930bcf81ce8"}
Sep 10 16:11:49.508095 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:49.507987    2527 scope.go:117] "RemoveContainer" containerID="aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d"
Sep 10 16:11:49.518019 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:49.518000    2527 scope.go:117] "RemoveContainer" containerID="aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d"
Sep 10 16:11:49.518358 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:11:49.518337    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d\": container with ID starting with aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d not found: ID does not exist" containerID="aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d"
Sep 10 16:11:49.518413 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:49.518368    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d"} err="failed to get container status \"aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d\": rpc error: code = NotFound desc = could not find container \"aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d\": container with ID starting with aa031c71efaa4571c2179ad4bcc507653a7c4f23114dd54c0093e92194d9db2d not found: ID does not exist"
Sep 10 16:11:49.536599 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:49.536559    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-vp468"]
Sep 10 16:11:49.544939 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:49.544904    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-vp468"]
Sep 10 16:11:50.563821 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:11:50.563788    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d79009a5-1cda-4084-98c0-c9c90fdbc564" path="/var/lib/kubelet/pods/d79009a5-1cda-4084-98c0-c9c90fdbc564/volumes"
Sep 10 16:12:50.403804 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.403764    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69fdcbc57-5f5bc"]
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404165    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9923e511-2ba4-4aae-9ec7-11ab3c8bfe78" containerName="payload-pre-processing"
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404182    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="9923e511-2ba4-4aae-9ec7-11ab3c8bfe78" containerName="payload-pre-processing"
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404201    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b3ab5b-def6-41f1-a7a7-2494419781ee" containerName="maas-api"
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404210    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b3ab5b-def6-41f1-a7a7-2494419781ee" containerName="maas-api"
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404223    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" containerName="maas-api"
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404229    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" containerName="maas-api"
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404235    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d79009a5-1cda-4084-98c0-c9c90fdbc564" containerName="manager"
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404239    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="d79009a5-1cda-4084-98c0-c9c90fdbc564" containerName="manager"
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404245    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5956aa90-2b25-4fef-af35-2bf5486f6ef8" containerName="maas-api"
Sep 10 16:12:50.404246 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404249    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="5956aa90-2b25-4fef-af35-2bf5486f6ef8" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404258    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="438518e2-d5b4-451a-a108-1eb8ac389dc6" containerName="payload-processing"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404263    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="438518e2-d5b4-451a-a108-1eb8ac389dc6" containerName="payload-processing"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404269    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3deb3119-081a-4839-a4ef-f0f0c4868d44" containerName="payload-pre-processing"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404274    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="3deb3119-081a-4839-a4ef-f0f0c4868d44" containerName="payload-pre-processing"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404281    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21e5382c-ab31-4fd4-a28d-e0823c47220d" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404287    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="21e5382c-ab31-4fd4-a28d-e0823c47220d" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404299    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="018cae23-2e24-4116-a13b-a9db57cdcd8e" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404304    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="018cae23-2e24-4116-a13b-a9db57cdcd8e" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404367    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="47b3ab5b-def6-41f1-a7a7-2494419781ee" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404374    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="438518e2-d5b4-451a-a108-1eb8ac389dc6" containerName="payload-processing"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404382    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="21e5382c-ab31-4fd4-a28d-e0823c47220d" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404388    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6b14fbb-7835-4c6b-8d6c-d7b980d041ff" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404393    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="018cae23-2e24-4116-a13b-a9db57cdcd8e" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404398    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="5956aa90-2b25-4fef-af35-2bf5486f6ef8" containerName="maas-api"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404403    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="3deb3119-081a-4839-a4ef-f0f0c4868d44" containerName="payload-pre-processing"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404408    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="9923e511-2ba4-4aae-9ec7-11ab3c8bfe78" containerName="payload-pre-processing"
Sep 10 16:12:50.404613 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.404414    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="d79009a5-1cda-4084-98c0-c9c90fdbc564" containerName="manager"
Sep 10 16:12:50.407445 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.407424    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.410008 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.409989    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 16:12:50.410100 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.410087    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 16:12:50.410852 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.410835    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bxbnj\""
Sep 10 16:12:50.410904 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.410854    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 16:12:50.442663 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.442631    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-5f5bc"]
Sep 10 16:12:50.464641 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.464610    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-metrics-certs\") pod \"maas-controller-69fdcbc57-5f5bc\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") " pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.464833 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.464667    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-495p9\" (UniqueName: \"kubernetes.io/projected/3f6b00c1-464e-441f-ad86-257388865aca-kube-api-access-495p9\") pod \"maas-controller-69fdcbc57-5f5bc\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") " pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.464833 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.464702    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-webhook-cert\") pod \"maas-controller-69fdcbc57-5f5bc\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") " pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.565991 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.565952    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-metrics-certs\") pod \"maas-controller-69fdcbc57-5f5bc\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") " pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.566232 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.566032    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-495p9\" (UniqueName: \"kubernetes.io/projected/3f6b00c1-464e-441f-ad86-257388865aca-kube-api-access-495p9\") pod \"maas-controller-69fdcbc57-5f5bc\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") " pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.566232 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.566074    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-webhook-cert\") pod \"maas-controller-69fdcbc57-5f5bc\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") " pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.568674 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.568640    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-webhook-cert\") pod \"maas-controller-69fdcbc57-5f5bc\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") " pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.568815 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.568765    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-metrics-certs\") pod \"maas-controller-69fdcbc57-5f5bc\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") " pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.580772 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.580739    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-495p9\" (UniqueName: \"kubernetes.io/projected/3f6b00c1-464e-441f-ad86-257388865aca-kube-api-access-495p9\") pod \"maas-controller-69fdcbc57-5f5bc\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") " pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:50.717982 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:50.717889    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:51.012229 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:51.012200    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-5f5bc"]
Sep 10 16:12:51.014445 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:12:51.014402    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6b00c1_464e_441f_ad86_257388865aca.slice/crio-648f28fd5affc1ba7fa6ea732a4d0cf6ba052012552dda996d213c089df2e838 WatchSource:0}: Error finding container 648f28fd5affc1ba7fa6ea732a4d0cf6ba052012552dda996d213c089df2e838: Status 404 returned error can't find the container with id 648f28fd5affc1ba7fa6ea732a4d0cf6ba052012552dda996d213c089df2e838
Sep 10 16:12:51.015796 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:51.015773    2527 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:12:51.825168 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:51.825111    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-5f5bc" event={"ID":"3f6b00c1-464e-441f-ad86-257388865aca","Type":"ContainerStarted","Data":"648f28fd5affc1ba7fa6ea732a4d0cf6ba052012552dda996d213c089df2e838"}
Sep 10 16:12:52.831477 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:52.831443    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-5f5bc" event={"ID":"3f6b00c1-464e-441f-ad86-257388865aca","Type":"ContainerStarted","Data":"8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7"}
Sep 10 16:12:52.831901 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:52.831579    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:12:52.877861 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:12:52.877806    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69fdcbc57-5f5bc" podStartSLOduration=2.116461331 podStartE2EDuration="2.877786907s" totalImagesPullingTime="761.325576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:12:52.877079116 +0000 UTC m=+1868.793203991" watchObservedRunningTime="2026-09-10 16:12:52.877786907 +0000 UTC m=+1868.793911787"
Sep 10 16:13:03.839286 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:03.839251    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:13:15.341009 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.340924    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-5f5bc"]
Sep 10 16:13:15.341478 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.341186    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-69fdcbc57-5f5bc" podUID="3f6b00c1-464e-441f-ad86-257388865aca" containerName="manager" containerID="cri-o://8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7" gracePeriod=30
Sep 10 16:13:15.603861 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.603775    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:13:15.608915 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.608884    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-495p9\" (UniqueName: \"kubernetes.io/projected/3f6b00c1-464e-441f-ad86-257388865aca-kube-api-access-495p9\") pod \"3f6b00c1-464e-441f-ad86-257388865aca\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") "
Sep 10 16:13:15.609109 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.608972    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-metrics-certs\") pod \"3f6b00c1-464e-441f-ad86-257388865aca\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") "
Sep 10 16:13:15.609109 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.609018    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-webhook-cert\") pod \"3f6b00c1-464e-441f-ad86-257388865aca\" (UID: \"3f6b00c1-464e-441f-ad86-257388865aca\") "
Sep 10 16:13:15.611265 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.611227    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3f6b00c1-464e-441f-ad86-257388865aca" (UID: "3f6b00c1-464e-441f-ad86-257388865aca"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:13:15.611265 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.611245    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3f6b00c1-464e-441f-ad86-257388865aca" (UID: "3f6b00c1-464e-441f-ad86-257388865aca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:13:15.611418 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.611227    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f6b00c1-464e-441f-ad86-257388865aca-kube-api-access-495p9" (OuterVolumeSpecName: "kube-api-access-495p9") pod "3f6b00c1-464e-441f-ad86-257388865aca" (UID: "3f6b00c1-464e-441f-ad86-257388865aca"). InnerVolumeSpecName "kube-api-access-495p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:13:15.710467 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.710417    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-495p9\" (UniqueName: \"kubernetes.io/projected/3f6b00c1-464e-441f-ad86-257388865aca-kube-api-access-495p9\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:13:15.710467 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.710457    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:13:15.710467 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.710467    2527 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f6b00c1-464e-441f-ad86-257388865aca-webhook-cert\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:13:15.952500 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.952409    2527 generic.go:356] "Generic (PLEG): container finished" podID="3f6b00c1-464e-441f-ad86-257388865aca" containerID="8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7" exitCode=0
Sep 10 16:13:15.952500 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.952457    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-5f5bc"
Sep 10 16:13:15.952696 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.952498    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-5f5bc" event={"ID":"3f6b00c1-464e-441f-ad86-257388865aca","Type":"ContainerDied","Data":"8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7"}
Sep 10 16:13:15.952696 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.952558    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-5f5bc" event={"ID":"3f6b00c1-464e-441f-ad86-257388865aca","Type":"ContainerDied","Data":"648f28fd5affc1ba7fa6ea732a4d0cf6ba052012552dda996d213c089df2e838"}
Sep 10 16:13:15.952696 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.952581    2527 scope.go:117] "RemoveContainer" containerID="8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7"
Sep 10 16:13:15.962335 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.962312    2527 scope.go:117] "RemoveContainer" containerID="8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7"
Sep 10 16:13:15.962675 ip-10-0-136-61 kubenswrapper[2527]: E0910 16:13:15.962655    2527 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7\": container with ID starting with 8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7 not found: ID does not exist" containerID="8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7"
Sep 10 16:13:15.962730 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.962685    2527 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7"} err="failed to get container status \"8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7\": rpc error: code = NotFound desc = could not find container \"8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7\": container with ID starting with 8f9efefb83ff33605a930128733b77729b7f2950941a29699a5c8bbf2b4310b7 not found: ID does not exist"
Sep 10 16:13:15.975313 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.975272    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-5f5bc"]
Sep 10 16:13:15.979573 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:15.979540    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-5f5bc"]
Sep 10 16:13:16.564908 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.564872    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f6b00c1-464e-441f-ad86-257388865aca" path="/var/lib/kubelet/pods/3f6b00c1-464e-441f-ad86-257388865aca/volumes"
Sep 10 16:13:16.737169 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.737121    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69fdcbc57-98skq"]
Sep 10 16:13:16.737564 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.737551    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f6b00c1-464e-441f-ad86-257388865aca" containerName="manager"
Sep 10 16:13:16.737608 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.737566    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f6b00c1-464e-441f-ad86-257388865aca" containerName="manager"
Sep 10 16:13:16.737644 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.737637    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f6b00c1-464e-441f-ad86-257388865aca" containerName="manager"
Sep 10 16:13:16.742242 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.742216    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:16.744721 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.744698    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 16:13:16.745488 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.745467    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bxbnj\""
Sep 10 16:13:16.745604 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.745504    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 16:13:16.745604 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.745520    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 16:13:16.752784 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.752758    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-98skq"]
Sep 10 16:13:16.820362 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.820258    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lfvm\" (UniqueName: \"kubernetes.io/projected/1b9c62ef-7102-4a73-9dbe-e12ad2b92502-kube-api-access-5lfvm\") pod \"maas-controller-69fdcbc57-98skq\" (UID: \"1b9c62ef-7102-4a73-9dbe-e12ad2b92502\") " pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:16.820362 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.820331    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b9c62ef-7102-4a73-9dbe-e12ad2b92502-webhook-cert\") pod \"maas-controller-69fdcbc57-98skq\" (UID: \"1b9c62ef-7102-4a73-9dbe-e12ad2b92502\") " pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:16.820569 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.820370    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b9c62ef-7102-4a73-9dbe-e12ad2b92502-metrics-certs\") pod \"maas-controller-69fdcbc57-98skq\" (UID: \"1b9c62ef-7102-4a73-9dbe-e12ad2b92502\") " pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:16.921061 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.921017    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lfvm\" (UniqueName: \"kubernetes.io/projected/1b9c62ef-7102-4a73-9dbe-e12ad2b92502-kube-api-access-5lfvm\") pod \"maas-controller-69fdcbc57-98skq\" (UID: \"1b9c62ef-7102-4a73-9dbe-e12ad2b92502\") " pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:16.921286 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.921091    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b9c62ef-7102-4a73-9dbe-e12ad2b92502-webhook-cert\") pod \"maas-controller-69fdcbc57-98skq\" (UID: \"1b9c62ef-7102-4a73-9dbe-e12ad2b92502\") " pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:16.921286 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.921130    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b9c62ef-7102-4a73-9dbe-e12ad2b92502-metrics-certs\") pod \"maas-controller-69fdcbc57-98skq\" (UID: \"1b9c62ef-7102-4a73-9dbe-e12ad2b92502\") " pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:16.923639 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.923609    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b9c62ef-7102-4a73-9dbe-e12ad2b92502-webhook-cert\") pod \"maas-controller-69fdcbc57-98skq\" (UID: \"1b9c62ef-7102-4a73-9dbe-e12ad2b92502\") " pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:16.923639 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.923634    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b9c62ef-7102-4a73-9dbe-e12ad2b92502-metrics-certs\") pod \"maas-controller-69fdcbc57-98skq\" (UID: \"1b9c62ef-7102-4a73-9dbe-e12ad2b92502\") " pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:16.929698 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:16.929669    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lfvm\" (UniqueName: \"kubernetes.io/projected/1b9c62ef-7102-4a73-9dbe-e12ad2b92502-kube-api-access-5lfvm\") pod \"maas-controller-69fdcbc57-98skq\" (UID: \"1b9c62ef-7102-4a73-9dbe-e12ad2b92502\") " pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:17.054168 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:17.054101    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:17.337055 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:17.337028    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69fdcbc57-98skq"]
Sep 10 16:13:17.338964 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:13:17.338936    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9c62ef_7102_4a73_9dbe_e12ad2b92502.slice/crio-a369c5c19650e17688bc082e6995647c50b775639cdaf6d4338ac30058d69f0a WatchSource:0}: Error finding container a369c5c19650e17688bc082e6995647c50b775639cdaf6d4338ac30058d69f0a: Status 404 returned error can't find the container with id a369c5c19650e17688bc082e6995647c50b775639cdaf6d4338ac30058d69f0a
Sep 10 16:13:17.965862 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:17.965724    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-98skq" event={"ID":"1b9c62ef-7102-4a73-9dbe-e12ad2b92502","Type":"ContainerStarted","Data":"d05018bd7cd04a97b1925f3cbd47f3614106b7397666d9854e7f72f996521b42"}
Sep 10 16:13:17.966367 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:17.966006    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:13:17.966367 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:17.966044    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69fdcbc57-98skq" event={"ID":"1b9c62ef-7102-4a73-9dbe-e12ad2b92502","Type":"ContainerStarted","Data":"a369c5c19650e17688bc082e6995647c50b775639cdaf6d4338ac30058d69f0a"}
Sep 10 16:13:17.984029 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:17.983960    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69fdcbc57-98skq" podStartSLOduration=1.655967486 podStartE2EDuration="1.983937274s" totalImagesPullingTime="327.969788ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:13:17.982246233 +0000 UTC m=+1893.898371116" watchObservedRunningTime="2026-09-10 16:13:17.983937274 +0000 UTC m=+1893.900062154"
Sep 10 16:13:28.973891 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:13:28.973854    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69fdcbc57-98skq"
Sep 10 16:16:44.698604 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:16:44.698575    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 16:16:44.711393 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:16:44.711364    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 16:16:59.819804 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:16:59.819763    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"]
Sep 10 16:16:59.824191 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:16:59.824108    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:16:59.833434 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:16:59.833351    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"]
Sep 10 16:16:59.935286 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:16:59.935249    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe406d5e-681e-4abf-bfcc-cdb138925a41-metrics-certs\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:16:59.935510 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:16:59.935361    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe406d5e-681e-4abf-bfcc-cdb138925a41-maas-api-tls\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:16:59.935510 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:16:59.935378    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe406d5e-681e-4abf-bfcc-cdb138925a41-openshift-service-ca\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:16:59.935510 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:16:59.935406    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2cwk\" (UniqueName: \"kubernetes.io/projected/fe406d5e-681e-4abf-bfcc-cdb138925a41-kube-api-access-q2cwk\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.036872 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.036838    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe406d5e-681e-4abf-bfcc-cdb138925a41-maas-api-tls\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.036872 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.036875    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe406d5e-681e-4abf-bfcc-cdb138925a41-openshift-service-ca\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.037215 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.036905    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2cwk\" (UniqueName: \"kubernetes.io/projected/fe406d5e-681e-4abf-bfcc-cdb138925a41-kube-api-access-q2cwk\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.037215 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.036939    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe406d5e-681e-4abf-bfcc-cdb138925a41-metrics-certs\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.037576 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.037539    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe406d5e-681e-4abf-bfcc-cdb138925a41-openshift-service-ca\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.039540 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.039508    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe406d5e-681e-4abf-bfcc-cdb138925a41-maas-api-tls\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.039667 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.039551    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe406d5e-681e-4abf-bfcc-cdb138925a41-metrics-certs\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.044696 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.044672    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2cwk\" (UniqueName: \"kubernetes.io/projected/fe406d5e-681e-4abf-bfcc-cdb138925a41-kube-api-access-q2cwk\") pod \"maas-api-79df8fbc7f-fd87t\" (UID: \"fe406d5e-681e-4abf-bfcc-cdb138925a41\") " pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.140920 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.140814    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:00.423125 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:00.423096    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"]
Sep 10 16:17:00.425539 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:17:00.425509    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe406d5e_681e_4abf_bfcc_cdb138925a41.slice/crio-d10a62af6bde6f7400237e91bfe984313ef958c8d1b829bfe3c271d1ecc1b27c WatchSource:0}: Error finding container d10a62af6bde6f7400237e91bfe984313ef958c8d1b829bfe3c271d1ecc1b27c: Status 404 returned error can't find the container with id d10a62af6bde6f7400237e91bfe984313ef958c8d1b829bfe3c271d1ecc1b27c
Sep 10 16:17:01.129382 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:01.129342    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t" event={"ID":"fe406d5e-681e-4abf-bfcc-cdb138925a41","Type":"ContainerStarted","Data":"05cf333c9e9595f9b21933e394e946f48cfdeb2592fd33634afe31990b4d8bcc"}
Sep 10 16:17:01.129798 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:01.129403    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t" event={"ID":"fe406d5e-681e-4abf-bfcc-cdb138925a41","Type":"ContainerStarted","Data":"d10a62af6bde6f7400237e91bfe984313ef958c8d1b829bfe3c271d1ecc1b27c"}
Sep 10 16:17:01.129798 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:01.129453    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:01.149566 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:01.149434    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t" podStartSLOduration=1.694562641 podStartE2EDuration="2.149412691s" totalImagesPullingTime="454.85005ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:01.147807151 +0000 UTC m=+2117.063932069" watchObservedRunningTime="2026-09-10 16:17:01.149412691 +0000 UTC m=+2117.065537569"
Sep 10 16:17:07.141201 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:07.141163    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79df8fbc7f-fd87t"
Sep 10 16:17:15.002410 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:15.002369    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6689875f55-k25tg"]
Sep 10 16:17:15.002926 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:15.002656    2527 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg" podUID="bb39753e-59c7-4597-82be-dad7fb95eba3" containerName="maas-api" containerID="cri-o://fb919b4f3517bcfa8423b2e75c0d0595bb995029537b5bf73e26426841143c1d" gracePeriod=30
Sep 10 16:17:16.211205 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.211166    2527 generic.go:356] "Generic (PLEG): container finished" podID="bb39753e-59c7-4597-82be-dad7fb95eba3" containerID="fb919b4f3517bcfa8423b2e75c0d0595bb995029537b5bf73e26426841143c1d" exitCode=0
Sep 10 16:17:16.211629 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.211251    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg" event={"ID":"bb39753e-59c7-4597-82be-dad7fb95eba3","Type":"ContainerDied","Data":"fb919b4f3517bcfa8423b2e75c0d0595bb995029537b5bf73e26426841143c1d"}
Sep 10 16:17:16.267912 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.267880    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 16:17:16.403356 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.403245    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb39753e-59c7-4597-82be-dad7fb95eba3-openshift-service-ca\") pod \"bb39753e-59c7-4597-82be-dad7fb95eba3\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") "
Sep 10 16:17:16.403356 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.403352    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-maas-api-tls\") pod \"bb39753e-59c7-4597-82be-dad7fb95eba3\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") "
Sep 10 16:17:16.403602 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.403402    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-metrics-certs\") pod \"bb39753e-59c7-4597-82be-dad7fb95eba3\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") "
Sep 10 16:17:16.403602 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.403426    2527 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27cgw\" (UniqueName: \"kubernetes.io/projected/bb39753e-59c7-4597-82be-dad7fb95eba3-kube-api-access-27cgw\") pod \"bb39753e-59c7-4597-82be-dad7fb95eba3\" (UID: \"bb39753e-59c7-4597-82be-dad7fb95eba3\") "
Sep 10 16:17:16.403732 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.403699    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb39753e-59c7-4597-82be-dad7fb95eba3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb39753e-59c7-4597-82be-dad7fb95eba3" (UID: "bb39753e-59c7-4597-82be-dad7fb95eba3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:17:16.405762 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.405726    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb39753e-59c7-4597-82be-dad7fb95eba3" (UID: "bb39753e-59c7-4597-82be-dad7fb95eba3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:17:16.405762 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.405743    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb39753e-59c7-4597-82be-dad7fb95eba3-kube-api-access-27cgw" (OuterVolumeSpecName: "kube-api-access-27cgw") pod "bb39753e-59c7-4597-82be-dad7fb95eba3" (UID: "bb39753e-59c7-4597-82be-dad7fb95eba3"). InnerVolumeSpecName "kube-api-access-27cgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:17:16.405901 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.405788    2527 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bb39753e-59c7-4597-82be-dad7fb95eba3" (UID: "bb39753e-59c7-4597-82be-dad7fb95eba3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:17:16.505055 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.504999    2527 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb39753e-59c7-4597-82be-dad7fb95eba3-openshift-service-ca\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:17:16.505055 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.505042    2527 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-maas-api-tls\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:17:16.505055 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.505052    2527 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb39753e-59c7-4597-82be-dad7fb95eba3-metrics-certs\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:17:16.505055 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:16.505060    2527 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27cgw\" (UniqueName: \"kubernetes.io/projected/bb39753e-59c7-4597-82be-dad7fb95eba3-kube-api-access-27cgw\") on node \"ip-10-0-136-61.ec2.internal\" DevicePath \"\""
Sep 10 16:17:17.217677 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:17.217645    2527 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg"
Sep 10 16:17:17.218114 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:17.217644    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6689875f55-k25tg" event={"ID":"bb39753e-59c7-4597-82be-dad7fb95eba3","Type":"ContainerDied","Data":"d45d2baf520948e6889ef813b4c27f3f98745fddeec3a7a273c5d64f2f89d423"}
Sep 10 16:17:17.218114 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:17.217785    2527 scope.go:117] "RemoveContainer" containerID="fb919b4f3517bcfa8423b2e75c0d0595bb995029537b5bf73e26426841143c1d"
Sep 10 16:17:17.256182 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:17.256125    2527 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6689875f55-k25tg"]
Sep 10 16:17:17.265151 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:17.265108    2527 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6689875f55-k25tg"]
Sep 10 16:17:18.563879 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:18.563829    2527 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb39753e-59c7-4597-82be-dad7fb95eba3" path="/var/lib/kubelet/pods/bb39753e-59c7-4597-82be-dad7fb95eba3/volumes"
Sep 10 16:17:35.179833 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:35.179799    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69fdcbc57-98skq_1b9c62ef-7102-4a73-9dbe-e12ad2b92502/manager/0.log"
Sep 10 16:17:35.802151 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:35.802115    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-97f7948ff-z25b9_4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe/copy-manifests/0.log"
Sep 10 16:17:35.818784 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:35.818749    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-97f7948ff-z25b9_4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe/manager/0.log"
Sep 10 16:17:36.179227 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:36.179123    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-79df8fbc7f-fd87t_fe406d5e-681e-4abf-bfcc-cdb138925a41/maas-api/0.log"
Sep 10 16:17:36.624932 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:36.624901    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-jxtlx_9ac0f436-4b90-40f0-bb12-356c8af39150/postgres/0.log"
Sep 10 16:17:37.506997 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:37.506964    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-t6nlf_d3f96f25-58fd-4b6a-87e6-1c848688cafb/manager/0.log"
Sep 10 16:17:37.628669 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:37.628640    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-79974_d1e5ed87-b513-4c07-8597-1d4170632bcc/manager/0.log"
Sep 10 16:17:39.201378 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:39.201338    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-88prc_e8cb2ee3-5670-4eb0-b31d-afa362698dd5/payload-processing/0.log"
Sep 10 16:17:39.319554 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:39.319520    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5cf6fc6776-qgwn4_e6549d43-a327-43b8-b3a9-0c71532c4985/router/0.log"
Sep 10 16:17:39.668231 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:39.668201    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-6vph5_48825b55-da89-4817-bc4d-dcecf274ab67/main/0.log"
Sep 10 16:17:39.890647 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:39.890618    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-wfdpj_8d352c83-b914-4ad8-b0ff-5d4bb8c6a39d/main/0.log"
Sep 10 16:17:40.006458 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:40.006363    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-q9lhv_b234007a-b9d3-491a-8e8e-2080339aa97f/main/0.log"
Sep 10 16:17:49.017429 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:49.017394    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z9ct5_f0d7f839-f1d5-4657-aa6c-5b19b248622b/global-pull-secret-syncer/0.log"
Sep 10 16:17:49.155514 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:49.155479    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v96j6_ca288a4f-6d91-475a-bdd0-22fb9c6c8361/konnectivity-agent/0.log"
Sep 10 16:17:49.209829 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:49.209796    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-61.ec2.internal_9c9407f7c27661a1f8778abcf84b0b26/haproxy/0.log"
Sep 10 16:17:53.625675 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:53.625584    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-t6nlf_d3f96f25-58fd-4b6a-87e6-1c848688cafb/manager/0.log"
Sep 10 16:17:53.652851 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:53.652819    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-79974_d1e5ed87-b513-4c07-8597-1d4170632bcc/manager/0.log"
Sep 10 16:17:55.385611 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:55.385577    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-tggpc_137e05f8-fbb8-45bb-af7e-418ecd5f1f51/cluster-monitoring-operator/0.log"
Sep 10 16:17:55.577707 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:55.577679    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5z6dv_5e19361d-b37c-46a6-97a2-24c2c26b3eb0/node-exporter/0.log"
Sep 10 16:17:55.608899 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:55.608871    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5z6dv_5e19361d-b37c-46a6-97a2-24c2c26b3eb0/kube-rbac-proxy/0.log"
Sep 10 16:17:55.637352 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:55.637258    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5z6dv_5e19361d-b37c-46a6-97a2-24c2c26b3eb0/init-textfile/0.log"
Sep 10 16:17:56.236555 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:56.236525    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-lt4pw_ebdbdd63-3263-4d90-8cec-8d2b22bdb693/prometheus-operator-admission-webhook/0.log"
Sep 10 16:17:57.993230 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:57.993177    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-d47zd_078c1103-50b4-4a77-ba94-0d1e164c5b41/networking-console-plugin/0.log"
Sep 10 16:17:58.616932 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:58.616899    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/1.log"
Sep 10 16:17:58.621511 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:58.621486    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-595p9_d30a715d-e7b9-4e9e-b311-2675c91df3df/console-operator/2.log"
Sep 10 16:17:59.200776 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.200743    2527 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"]
Sep 10 16:17:59.201208 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.201192    2527 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb39753e-59c7-4597-82be-dad7fb95eba3" containerName="maas-api"
Sep 10 16:17:59.201261 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.201211    2527 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb39753e-59c7-4597-82be-dad7fb95eba3" containerName="maas-api"
Sep 10 16:17:59.201298 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.201280    2527 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb39753e-59c7-4597-82be-dad7fb95eba3" containerName="maas-api"
Sep 10 16:17:59.204516 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.204492    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.209195 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.209169    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t86vd\"/\"kube-root-ca.crt\""
Sep 10 16:17:59.209624 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.209605    2527 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t86vd\"/\"openshift-service-ca.crt\""
Sep 10 16:17:59.209723 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.209612    2527 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t86vd\"/\"default-dockercfg-b47qk\""
Sep 10 16:17:59.220766 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.220734    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"]
Sep 10 16:17:59.259893 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.259865    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-745779c994-vf2p6_3b3f9eb2-cdac-468d-bf2d-c0faf0f03b79/console/0.log"
Sep 10 16:17:59.318557 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.318523    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-v5z46_b6d5f4ab-9262-4c92-a259-a2ef772863bd/download-server/0.log"
Sep 10 16:17:59.326837 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.326808    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-proc\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.326971 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.326844    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-lib-modules\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.326971 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.326875    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-sys\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.326971 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.326944    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-podres\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.327079 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.327036    2527 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6fwb\" (UniqueName: \"kubernetes.io/projected/6488307d-dc2b-441e-978a-f1debc6c95ad-kube-api-access-p6fwb\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.428533 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.428482    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-sys\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.428533 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.428536    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-podres\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.428763 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.428605    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6fwb\" (UniqueName: \"kubernetes.io/projected/6488307d-dc2b-441e-978a-f1debc6c95ad-kube-api-access-p6fwb\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.428763 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.428618    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-sys\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.428763 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.428686    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-proc\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.428763 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.428701    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-podres\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.428763 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.428716    2527 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-lib-modules\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.428763 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.428757    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-proc\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.428955 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.428849    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6488307d-dc2b-441e-978a-f1debc6c95ad-lib-modules\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.436826 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.436799    2527 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6fwb\" (UniqueName: \"kubernetes.io/projected/6488307d-dc2b-441e-978a-f1debc6c95ad-kube-api-access-p6fwb\") pod \"perf-node-gather-daemonset-z8qvn\" (UID: \"6488307d-dc2b-441e-978a-f1debc6c95ad\") " pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.515621 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.515522    2527 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:17:59.809693 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:17:59.809660    2527 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"]
Sep 10 16:17:59.813077 ip-10-0-136-61 kubenswrapper[2527]: W0910 16:17:59.813045    2527 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6488307d_dc2b_441e_978a_f1debc6c95ad.slice/crio-7e50a163236cb6f755f2012b6d05bad32dec1325f9531fc618cfba93ac638059 WatchSource:0}: Error finding container 7e50a163236cb6f755f2012b6d05bad32dec1325f9531fc618cfba93ac638059: Status 404 returned error can't find the container with id 7e50a163236cb6f755f2012b6d05bad32dec1325f9531fc618cfba93ac638059
Sep 10 16:18:00.100489 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:00.100403    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-58bxj_c8fd6930-9ef8-4269-8e75-370c3f2d4deb/volume-data-source-validator/0.log"
Sep 10 16:18:00.448757 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:00.448645    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn" event={"ID":"6488307d-dc2b-441e-978a-f1debc6c95ad","Type":"ContainerStarted","Data":"10ca699c9214260a9cc6c691780265fff967b5f597cfa8d5755f6dfdb326b04e"}
Sep 10 16:18:00.448757 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:00.448707    2527 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn" event={"ID":"6488307d-dc2b-441e-978a-f1debc6c95ad","Type":"ContainerStarted","Data":"7e50a163236cb6f755f2012b6d05bad32dec1325f9531fc618cfba93ac638059"}
Sep 10 16:18:00.448757 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:00.448736    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:18:00.470153 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:00.470086    2527 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn" podStartSLOduration=1.4700672909999999 podStartE2EDuration="1.470067291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:00.467401987 +0000 UTC m=+2176.383526879" watchObservedRunningTime="2026-09-10 16:18:00.470067291 +0000 UTC m=+2176.386192167"
Sep 10 16:18:01.282402 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:01.282369    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wnlqg_909c47d6-65aa-4f4f-bfc3-da05c15bc3c9/dns/0.log"
Sep 10 16:18:01.338101 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:01.338068    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wnlqg_909c47d6-65aa-4f4f-bfc3-da05c15bc3c9/kube-rbac-proxy/0.log"
Sep 10 16:18:01.488500 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:01.488468    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4djmx_54d5a997-3142-4442-a345-298d70f46804/dns-node-resolver/0.log"
Sep 10 16:18:02.184584 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:02.184553    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xfqhc_193f19ff-ce68-42c7-929f-f26b2c5edde7/node-ca/0.log"
Sep 10 16:18:03.960780 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:03.960736    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-88prc_e8cb2ee3-5670-4eb0-b31d-afa362698dd5/payload-processing/0.log"
Sep 10 16:18:03.992395 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:03.992362    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5cf6fc6776-qgwn4_e6549d43-a327-43b8-b3a9-0c71532c4985/router/0.log"
Sep 10 16:18:04.957714 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:04.957681    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h5mql_2120541f-3d27-4e5e-8439-374a8d237e26/serve-healthcheck-canary/0.log"
Sep 10 16:18:05.571700 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:05.571663    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-5gl8t_8101fd3c-1dcc-4099-88af-6581442ee257/insights-operator/0.log"
Sep 10 16:18:05.572404 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:05.572388    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-5gl8t_8101fd3c-1dcc-4099-88af-6581442ee257/insights-operator/1.log"
Sep 10 16:18:05.606814 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:05.606780    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gqclh_ee6b2a6f-4882-4cce-96cf-a8225b5b864e/kube-rbac-proxy/0.log"
Sep 10 16:18:05.640449 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:05.640425    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gqclh_ee6b2a6f-4882-4cce-96cf-a8225b5b864e/exporter/0.log"
Sep 10 16:18:05.675713 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:05.675680    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gqclh_ee6b2a6f-4882-4cce-96cf-a8225b5b864e/extractor/0.log"
Sep 10 16:18:06.464445 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:06.464411    2527 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t86vd/perf-node-gather-daemonset-z8qvn"
Sep 10 16:18:08.928424 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:08.928394    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69fdcbc57-98skq_1b9c62ef-7102-4a73-9dbe-e12ad2b92502/manager/0.log"
Sep 10 16:18:09.281709 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:09.281674    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-97f7948ff-z25b9_4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe/manager/0.log"
Sep 10 16:18:09.300779 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:09.300755    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-97f7948ff-z25b9_4cb8cc7f-5a04-4be0-9b7d-708e8423a6fe/copy-manifests/0.log"
Sep 10 16:18:10.115963 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:10.115935    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-88prc_e8cb2ee3-5670-4eb0-b31d-afa362698dd5/payload-processing/0.log"
Sep 10 16:18:10.137145 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:10.137111    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5cf6fc6776-qgwn4_e6549d43-a327-43b8-b3a9-0c71532c4985/router/0.log"
Sep 10 16:18:10.688748 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:10.688647    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-79dbb968b5-m588m_67657edb-0b99-4240-b865-6dd8a955ed13/manager/0.log"
Sep 10 16:18:12.338510 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:12.338470    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-88prc_e8cb2ee3-5670-4eb0-b31d-afa362698dd5/payload-processing/0.log"
Sep 10 16:18:12.365853 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:12.365801    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5cf6fc6776-qgwn4_e6549d43-a327-43b8-b3a9-0c71532c4985/router/0.log"
Sep 10 16:18:15.987312 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:15.987276    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8psxt_3df79d94-eaf2-4a4a-a368-8906ea79d488/migrator/0.log"
Sep 10 16:18:16.010950 ip-10-0-136-61 kubenswrapper[2527]: I0910 16:18:16.010915    2527 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8psxt_3df79d94-eaf2-4a4a-a368-8906ea79d488/graceful-termination/0.log"