Jul 30 16:34:00.989806 ip-10-0-135-111 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 30 16:34:00.989818 ip-10-0-135-111 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 30 16:34:00.989827 ip-10-0-135-111 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 30 16:34:00.990135 ip-10-0-135-111 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 30 16:34:11.132727 ip-10-0-135-111 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 30 16:34:11.132748 ip-10-0-135-111 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ffea3095afa7431ea77697df8c9c78da --
Jul 30 16:36:12.279942 ip-10-0-135-111 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 16:36:12.699211 ip-10-0-135-111 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 16:36:12.699211 ip-10-0-135-111 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 16:36:12.699211 ip-10-0-135-111 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 16:36:12.699211 ip-10-0-135-111 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 16:36:12.699211 ip-10-0-135-111 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 16:36:12.699914 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.699264    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 16:36:12.703232 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703200    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:12.703232 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703230    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703241    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703247    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703254    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703259    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703263    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703268    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703272    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703277    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703283    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703289    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703294    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703298    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703310    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703314    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703319    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703323    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703328    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703332    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703337    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:12.703385 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703342    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703346    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703351    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703355    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703360    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703364    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703373    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703378    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703382    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703386    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703390    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703395    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703399    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703404    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703408    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703412    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703416    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703420    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703424    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703434    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:12.704063 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703438    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703442    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703446    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703451    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703454    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703458    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703918    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703942    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.703962    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704097    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704105    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704109    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704112    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704115    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704118    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704121    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704124    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704126    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704132    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704135    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:12.704575 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704138    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704143    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704148    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704152    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704156    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704159    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704163    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704166    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704169    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704172    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704175    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704178    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704181    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704184    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704187    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704190    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704192    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704195    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704197    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:12.705086 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704200    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704203    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704207    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704209    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704212    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704215    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704219    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704222    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704225    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704228    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704231    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704233    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704237    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704239    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704242    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704244    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704247    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704249    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:12.705536 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.704251    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705827    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705837    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705840    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705843    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705846    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705849    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705853    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705855    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705859    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705861    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705864    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705867    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705869    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705872    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705875    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705877    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705879    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705882    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705884    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:12.705985 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705887    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705889    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705892    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705894    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705896    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705899    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705901    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705906    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705909    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705913    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705916    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705918    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705920    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705924    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705927    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705929    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705932    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705934    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705937    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:12.706461 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705939    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705942    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705945    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705947    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705950    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705952    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705955    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705957    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705959    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705962    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705964    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705967    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705970    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705973    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705975    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705978    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705980    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705983    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705986    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705988    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:12.706937 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705991    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705994    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705996    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.705999    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706001    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706005    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706010    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706013    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706016    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706019    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706022    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706025    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706028    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706031    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706033    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706035    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706038    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706041    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706043    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706045    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:12.707439 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706048    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706050    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706053    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706055    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706058    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706061    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706063    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706066    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706068    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706070    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706073    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706076    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706080    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706083    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706085    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706088    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706090    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706093    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706096    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:12.707936 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706098    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706101    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706489    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706494    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706497    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706501    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706504    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706507    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706509    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706512    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706515    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706517    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706520    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706522    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706525    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706528    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706530    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706533    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706536    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706539    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:12.708395 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706541    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706544    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706546    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706549    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706551    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706556    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706560    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706563    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706565    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706568    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706571    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706573    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706576    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706578    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706581    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706583    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706586    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706588    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:12.708896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706592    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706596    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706598    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706600    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706618    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706620    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706623    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706626    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706628    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706631    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706633    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706636    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706638    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706641    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706644    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706646    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706649    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706651    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706654    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706656    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:12.709325 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706659    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706661    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706663    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706666    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706669    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706671    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706673    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706676    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706678    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706681    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706683    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706686    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706689    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706691    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706694    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706696    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706698    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706701    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706703    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706706    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:12.709819 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706708    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706711    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706713    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706716    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706718    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706721    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706724    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706726    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706729    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706732    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706735    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706737    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706739    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706742    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706744    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706748    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706754    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706757    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706760    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706763    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:12.710293 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706766    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706769    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.706771    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706846    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706854    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706864    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706871    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706877    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706882    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706887    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706891    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706894    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706898    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706901    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706904    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706908    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706911    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706913    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706916    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706920    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706924    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706928    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706931    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 16:36:12.710783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706934    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706937    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706941    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706945    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706948    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706953    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706956    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706959    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706963    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706965    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706969    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706976    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706980    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706983    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706986    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706989    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706993    2569 flags.go:64] FLAG: --enable-server="true"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.706996    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707003    2569 flags.go:64] FLAG: --event-burst="100"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707006    2569 flags.go:64] FLAG: --event-qps="50"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707009    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707012    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707015    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707020    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 16:36:12.711321 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707022    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707025    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707028    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707032    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707034    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707038    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707041    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707044    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707046    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707049    2569 flags.go:64] FLAG: --feature-gates=""
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707053    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707056    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707059    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707064    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707067    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707070    2569 flags.go:64] FLAG: --help="false"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707072    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-135-111.ec2.internal"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707076    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707079    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707082    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707085    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707088    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707091    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 16:36:12.711899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707094    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707097    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707100    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707102    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707105    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707108    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707111    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707115    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707118    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707121    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707124    2569 flags.go:64] FLAG: --lock-file=""
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707127    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707130    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707133    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707139    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707142    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707145    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707147    2569 flags.go:64] FLAG: --logging-format="text"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707150    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707154    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707156    2569 flags.go:64] FLAG: --manifest-url=""
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707159    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707166    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707169    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707173    2569 flags.go:64] FLAG: --max-pods="110"
Jul 30 16:36:12.712461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707176    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707179    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707182    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707185    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707188    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707191    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707194    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707201    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707205    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707208    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707211    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707214    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707219    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707222    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707226    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707229    2569 flags.go:64] FLAG: --port="10250"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707232    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707235    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0174827455626027a"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707238    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707241    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707243    2569 flags.go:64] FLAG: --register-node="true"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707246    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707250    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707253    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 30 16:36:12.713090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707257    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707260    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707263    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707266    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707269    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707272    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707276    2569 flags.go:64] FLAG: --runonce="false"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707279    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707282    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707285    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707288    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707290    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707293    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707297    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707299    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707302    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707305    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707309    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707312    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707315    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707318    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707320    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707327    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707329    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707332    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 16:36:12.713797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707336    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707339    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707342    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707344    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707347    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707350    2569 flags.go:64] FLAG: --v="2"
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707354    2569 flags.go:64] FLAG: --version="false"
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707359    2569 flags.go:64] FLAG: --vmodule=""
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707363    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.707366    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707499    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707504    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707507    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707512    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707515    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707518    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707521    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707525    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707529    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707532    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707535    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:12.714863 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707538    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707541    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707543    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707546    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707549    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707553    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707556    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707559    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707564    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707566    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707569    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707572    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707575    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707578    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707580    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707583    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707585    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707588    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707591    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:12.715734 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707594    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707596    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707598    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707601    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707615    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707619    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707622    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707624    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707627    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707629    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707632    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707635    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707638    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707640    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707643    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707645    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707648    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707650    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707653    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707655    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:12.716259 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707658    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707662    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707664    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707667    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707670    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707672    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707675    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707677    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707679    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707682    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707685    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707687    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707689    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707692    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707695    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707697    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707700    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707704    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707706    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707709    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:12.716818 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707711    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707714    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707716    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707719    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707721    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707724    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707726    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707729    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707731    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707734    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707737    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707739    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707743    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707747    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707750    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707753    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707756    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707758    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707760    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707763    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:12.717341 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707765    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707768    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707770    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707772    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707775    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707777    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707781    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707783    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.707786    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.708290    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.715812    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.715833    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.717638    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.717661    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717748    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:12.718144 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717756    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717761    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717765    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717769    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717774    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717778    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717782    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717788    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717792    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717796    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717800    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717804    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717808    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717813    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717817    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717824    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717829    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717834    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717839    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:12.718807 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717845    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717851    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717855    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717860    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717865    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717869    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717873    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717877    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717882    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717888    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717892    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717897    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717901    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717905    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717909    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717914    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717918    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717922    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717926    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717930    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:12.719393 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717935    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717940    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717944    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717948    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717952    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717956    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717961    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717965    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717970    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717974    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717978    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717983    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717987    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717991    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.717996    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718000    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718005    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718010    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718014    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718018    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:12.720168 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718022    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718026    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718031    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718035    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718039    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718045    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718052    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718056    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718060    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718064    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718068    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718072    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718076    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718080    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718084    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718088    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718092    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718096    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718100    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:12.721001 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718104    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718108    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718113    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718117    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718121    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718126    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718130    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718134    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718138    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718142    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718146    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718150    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718153    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718157    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718161    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718165    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718170    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718175    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718179    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:12.721509 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718184    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.718192    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718370    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718379    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718384    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718389    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718394    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718398    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718403    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718407    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718412    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718416    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718421    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718425    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718429    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 16:36:12.722217 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718433    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718437    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718441    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718446    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718451    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718455    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718461    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718465    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718469    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718473    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718477    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718481    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718485    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718489    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718493    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718498    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718503    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718508    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718512    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718517    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 16:36:12.722585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718524    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718528    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718532    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718537    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718540    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718544    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718548    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718553    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718557    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718561    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718565    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718569    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718573    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718577    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718581    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718586    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718590    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718594    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718599    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718619    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 16:36:12.723097 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718625    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718629    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718633    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718638    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718642    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718646    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718652    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718658    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718664    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718669    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718673    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718677    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718682    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718686    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718690    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718695    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718698    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718703    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718707    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 16:36:12.723571 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718711    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718718    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718724    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718728    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718732    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718737    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718742    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718745    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718750    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718754    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718758    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718762    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718766    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718771    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718775    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718779    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718783    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718787    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718791    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 16:36:12.724043 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718796    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718800    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718804    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718815    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718819    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718823    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718827    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:12.718831    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.718839    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.719014    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.721496    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.722914    2569 server.go:1019] "Starting client certificate rotation"
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.723015    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 16:36:12.724490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.723057    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 16:36:12.726143 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.726128    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 16:36:12.729137 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.729122    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 16:36:12.742203 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.742182    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 30 16:36:12.744214 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.744198    2569 log.go:25] "Validated CRI v1 image API"
Jul 30 16:36:12.746089 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.746071    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 16:36:12.749949 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.749928    2569 fs.go:135] Filesystem UUIDs: map[7787f824-1af4-4b0a-b19c-97357bde3211:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7acbabab-7eee-471a-968f-937b8d8605a2:/dev/nvme0n1p4]
Jul 30 16:36:12.749998 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.749949    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 16:36:12.751019 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.751002    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 16:36:12.755946 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.755832    2569 manager.go:216] Machine: {Timestamp:2026-07-30 16:36:12.754491216 +0000 UTC m=+0.356918832 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3199112 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2858e934fdb64ba190ea936f2c25c2 SystemUUID:ec2858e9-34fd-b64b-a190-ea936f2c25c2 BootID:ffea3095-afa7-431e-a776-97df8c9c78da Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a2:02:2d:54:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a2:02:2d:54:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:b5:39:d0:a1:f1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 16:36:12.755946 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.755939    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 16:36:12.756075 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.756064    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 16:36:12.759119 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.759092    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 16:36:12.759260 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.759122    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-111.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 16:36:12.760680 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.760667    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 16:36:12.760712 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.760683    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 16:36:12.760712 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.760691    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 16:36:12.761333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.761321    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 16:36:12.761957 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.761947    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 16:36:12.763632 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.763601    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 16:36:12.763764 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.763754    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 16:36:12.763802 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.763787    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 16:36:12.763802 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.763797    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 16:36:12.763865 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.763811    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 16:36:12.763865 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.763820    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 30 16:36:12.763865 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.763829    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 16:36:12.764221 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.764207    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 16:36:12.764258 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.764233    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 16:36:12.766736 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.766718    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 16:36:12.768415 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.768397    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 16:36:12.768507 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.768428    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 16:36:12.769878 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769858    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 16:36:12.769878 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769881    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769888    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769894    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769903    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769909    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769918    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769923    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769931    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769936    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769950    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 16:36:12.769986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.769956    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 16:36:12.771566 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.771555    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 16:36:12.771639 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.771568    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 16:36:12.773902 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.773876    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kqdtd"
Jul 30 16:36:12.774407 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.774378    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-111.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 16:36:12.774407 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.774393    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-111.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 16:36:12.774533 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.774412    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 16:36:12.774894 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.774883    2569 server.go:1268] "Started kubelet"
Jul 30 16:36:12.774986 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.774958    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 16:36:12.775031 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.774972    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 16:36:12.775077 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.775053    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 16:36:12.779820 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.779674    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 16:36:12.779805 ip-10-0-135-111 systemd[1]: Started Kubernetes Kubelet.
Jul 30 16:36:12.783313 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.782436    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 16:36:12.783574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.783521    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 16:36:12.784014 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.783996    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kqdtd"
Jul 30 16:36:12.787028 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.786076    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-111.ec2.internal.18c71e8e0fcb2f0e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-111.ec2.internal,UID:ip-10-0-135-111.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-111.ec2.internal,},FirstTimestamp:2026-07-30 16:36:12.774854414 +0000 UTC m=+0.377282104,LastTimestamp:2026-07-30 16:36:12.774854414 +0000 UTC m=+0.377282104,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-111.ec2.internal,}"
Jul 30 16:36:12.788739 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.788724    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 16:36:12.788801 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.788783    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 16:36:12.789035 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.788919    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:12.789235 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789208    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 16:36:12.789316 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789208    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 16:36:12.789316 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789311    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 16:36:12.789490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789472    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 16:36:12.789490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789482    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 16:36:12.789682 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789667    2569 factory.go:55] Registering systemd factory
Jul 30 16:36:12.789682 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789684    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 30 16:36:12.789946 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789924    2569 factory.go:153] Registering CRI-O factory
Jul 30 16:36:12.789946 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789944    2569 factory.go:223] Registration of the crio container factory successfully
Jul 30 16:36:12.790064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.789992    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 16:36:12.790064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.790011    2569 factory.go:103] Registering Raw factory
Jul 30 16:36:12.790064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.790025    2569 manager.go:1196] Started watching for new ooms in manager
Jul 30 16:36:12.790510 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.790497    2569 manager.go:349] Starting recovery of all containers
Jul 30 16:36:12.794584 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.794567    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:12.798635 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.798593    2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-111.ec2.internal\" not found" node="ip-10-0-135-111.ec2.internal"
Jul 30 16:36:12.801194 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.801173    2569 manager.go:354] Recovery completed
Jul 30 16:36:12.805709 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.805691    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:12.808216 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.808202    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:12.808292 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.808233    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:12.808292 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.808244    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:12.808889 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.808769    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 16:36:12.808889 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.808778    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:12.808889 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.808782    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 16:36:12.808889 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.808843    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 16:36:12.809534 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.809521    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:12.809574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.809547    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:12.809574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.809555    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:12.810364 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.810353    2569 policy_none.go:49] "None policy: Start"
Jul 30 16:36:12.810396 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.810369    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 16:36:12.810396 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.810379    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 16:36:12.811203 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.811193    2569 policy_none.go:47] "Start"
Jul 30 16:36:12.845506 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.845298    2569 manager.go:337] "Starting Device Plugin manager"
Jul 30 16:36:12.845634 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.845543    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 16:36:12.845634 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.845557    2569 server.go:87] "Starting device plugin registration server"
Jul 30 16:36:12.845871 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.845859    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 16:36:12.845901 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.845875    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 16:36:12.845999 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.845970    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 16:36:12.846119 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.846067    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 16:36:12.846119 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.846079    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 16:36:12.846629 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.846590    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 16:36:12.846708 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.846669    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:12.947462 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.947405    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:12.948487 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.948461    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:12.948604 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.948496    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:12.948604 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.948509    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:12.948604 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.948539    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-111.ec2.internal"
Jul 30 16:36:12.955393 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.955313    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-111.ec2.internal"
Jul 30 16:36:12.955393 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.955341    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-111.ec2.internal\": node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:12.975026 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.974994    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:12.992934 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.992888    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 16:36:12.994226 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.994198    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 16:36:12.994343 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.994244    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 16:36:12.994343 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.994266    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 16:36:12.994343 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:12.994308    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 16:36:12.999357 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:12.999338    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:13.077647 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.077597    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:13.094966 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.094931    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal"]
Jul 30 16:36:13.095078 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.095048    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:13.096149 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.096131    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:13.096261 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.096164    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:13.096261 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.096175    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:13.097581 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.097566    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:13.097755 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.097731    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.097801 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.097782    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:13.098514 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.098496    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:13.098619 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.098529    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:13.098619 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.098543    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:13.098619 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.098499    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:13.098619 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.098604    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:13.098827 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.098629    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:13.099616 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.099598    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.099727 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.099632    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 16:36:13.100540 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.100525    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 16:36:13.100634 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.100553    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 16:36:13.100634 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.100563    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeHasSufficientPID"
Jul 30 16:36:13.126701 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.126673    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-111.ec2.internal\" not found" node="ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.131447 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.131430    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-111.ec2.internal\" not found" node="ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.178013 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.177978    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:13.191690 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.191658    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47a748d50900f64c692e07ae4042b0bd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal\" (UID: \"47a748d50900f64c692e07ae4042b0bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.191690 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.191691    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47a748d50900f64c692e07ae4042b0bd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal\" (UID: \"47a748d50900f64c692e07ae4042b0bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.191883 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.191712    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c77bfff64908136b53abdd3eaff108d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-111.ec2.internal\" (UID: \"3c77bfff64908136b53abdd3eaff108d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.279087 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.278995    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:13.292412 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.292383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47a748d50900f64c692e07ae4042b0bd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal\" (UID: \"47a748d50900f64c692e07ae4042b0bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.292527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.292415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47a748d50900f64c692e07ae4042b0bd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal\" (UID: \"47a748d50900f64c692e07ae4042b0bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.292527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.292428    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47a748d50900f64c692e07ae4042b0bd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal\" (UID: \"47a748d50900f64c692e07ae4042b0bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.292527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.292447    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c77bfff64908136b53abdd3eaff108d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-111.ec2.internal\" (UID: \"3c77bfff64908136b53abdd3eaff108d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.292527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.292482    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47a748d50900f64c692e07ae4042b0bd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal\" (UID: \"47a748d50900f64c692e07ae4042b0bd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.292527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.292486    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c77bfff64908136b53abdd3eaff108d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-111.ec2.internal\" (UID: \"3c77bfff64908136b53abdd3eaff108d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.384897 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.384853    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:13.430755 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.430711    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.434413 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.434387    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:13.485700 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.485660    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:13.591194 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.591097    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:13.691326 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.691283    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:13.723533 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.723482    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 16:36:13.724127 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.723720    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 16:36:13.724127 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.723720    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 16:36:13.787754 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.787711    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 16:31:13 +0000 UTC" deadline="2028-04-13 09:21:16.884274396 +0000 UTC"
Jul 30 16:36:13.787754 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:13.787746    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14944h45m3.096532287s"
Jul 30 16:36:13.791395 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.791372    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:13.892154 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.892061    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:13.993183 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:13.993154    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:14.084859 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.084822    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:14.093630 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.093592    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-111.ec2.internal\" not found"
Jul 30 16:36:14.172979 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.172908    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:14.189196 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.189163    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:14.202479 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.202454    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 16:36:14.203699 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.203683    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal"
Jul 30 16:36:14.214520 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.214496    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 16:36:14.221704 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:14.221672    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a748d50900f64c692e07ae4042b0bd.slice/crio-5f9f5448f8e1744adea862d98b3aae8eabd9821d0d9e7838fd10061aaa93ed8b WatchSource:0}: Error finding container 5f9f5448f8e1744adea862d98b3aae8eabd9821d0d9e7838fd10061aaa93ed8b: Status 404 returned error can't find the container with id 5f9f5448f8e1744adea862d98b3aae8eabd9821d0d9e7838fd10061aaa93ed8b
Jul 30 16:36:14.222149 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:14.222128    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c77bfff64908136b53abdd3eaff108d.slice/crio-005f3edd6478964e3c8f3a612edc713cfbbbcd760ad38544a3dda5821f8a5fe7 WatchSource:0}: Error finding container 005f3edd6478964e3c8f3a612edc713cfbbbcd760ad38544a3dda5821f8a5fe7: Status 404 returned error can't find the container with id 005f3edd6478964e3c8f3a612edc713cfbbbcd760ad38544a3dda5821f8a5fe7
Jul 30 16:36:14.229784 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.229764    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 16:36:14.654950 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.654847    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:14.767410 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.767373    2569 apiserver.go:52] "Watching apiserver"
Jul 30 16:36:14.776923 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.776281    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 16:36:14.778406 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.778376    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-44szc","kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal","openshift-dns/node-resolver-79c74","openshift-multus/multus-additional-cni-plugins-ngd2j","openshift-multus/multus-dmktj","openshift-multus/network-metrics-daemon-qzc6g","openshift-network-operator/iptables-alerter-dqq42","kube-system/global-pull-secret-syncer-bwspl","kube-system/konnectivity-agent-rx4xb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg","openshift-cluster-node-tuning-operator/tuned-b6c2j","openshift-image-registry/node-ca-kw2nz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal","openshift-network-diagnostics/network-check-target-vd7hz"]
Jul 30 16:36:14.780443 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.780415    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.781530 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.781513    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.782776 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.782753    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.783653 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.783631    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9wkgx\""
Jul 30 16:36:14.783653 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.783647    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 16:36:14.783809 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.783668    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 16:36:14.783809 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.783647    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 16:36:14.784041 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.784024    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.784113 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.784056    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jd2l2\""
Jul 30 16:36:14.784880 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.784851    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 16:36:14.784989 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.784889    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 16:36:14.784989 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.784889    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 16:36:14.785212 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.785197    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.785360 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.785331    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m9skf\""
Jul 30 16:36:14.785594 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.785577    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 16:36:14.786212 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.786190    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:14.786312 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.786264    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:14.787434 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.787416    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:14.788267 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.788237    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 16:36:14.788267 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.788245    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 16:36:14.788519 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.788495    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zwlwg\""
Jul 30 16:36:14.788620 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.788532    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 16:36:14.788620 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.788547    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 16:36:14.788620 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.788586    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 16:36:14.788819 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.788702    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 16:36:14.788819 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.788707    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hpbmd\""
Jul 30 16:36:14.788997 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.788984    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:14.789084 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.789065    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:14.789127 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.789114    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 16:36:14.790173 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.790152    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 16:36:14.790342 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.790325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:14.790419 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.790400    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:14.790676 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.790661    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 16:36:14.791205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.791188    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xnsf6\""
Jul 30 16:36:14.792790 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.792765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:14.794452 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.794431    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.795676 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.795658    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 16:36:14.795778 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.795659    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 16:36:14.796077 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.796033    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mlhp2\""
Jul 30 16:36:14.796230 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.796213    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:14.797136 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.797093    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 16:36:14.798212 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798152    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 16:36:14.798212 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798152    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 16:36:14.798536 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798361    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7314837-02ee-4776-9fcf-430ad544afd9-iptables-alerter-script\") pod \"iptables-alerter-dqq42\" (UID: \"e7314837-02ee-4776-9fcf-430ad544afd9\") " pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.798536 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-tuned\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.798536 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-system-cni-dir\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.798536 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8406d9dc-3cba-47f7-9448-4ebc5d75972f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.798536 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798531    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp2xf\" (UniqueName: \"kubernetes.io/projected/264e7227-dd74-4d87-9f65-107983fb8538-kube-api-access-tp2xf\") pod \"node-resolver-79c74\" (UID: \"264e7227-dd74-4d87-9f65-107983fb8538\") " pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798555    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-hostroot\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798578    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-etc-kubernetes\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798603    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798637    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7314837-02ee-4776-9fcf-430ad544afd9-host-slash\") pod \"iptables-alerter-dqq42\" (UID: \"e7314837-02ee-4776-9fcf-430ad544afd9\") " pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798662    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-sysctl-d\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-run\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-host\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798742    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798766    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/264e7227-dd74-4d87-9f65-107983fb8538-tmp-dir\") pod \"node-resolver-79c74\" (UID: \"264e7227-dd74-4d87-9f65-107983fb8538\") " pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798794    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-os-release\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-systemd\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798833    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798844    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798888    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/264e7227-dd74-4d87-9f65-107983fb8538-hosts-file\") pod \"node-resolver-79c74\" (UID: \"264e7227-dd74-4d87-9f65-107983fb8538\") " pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798911    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-system-cni-dir\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798949    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-run-k8s-cni-cncf-io\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-var-lib-cni-bin\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.799095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.798996    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-var-lib-cni-multus\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-run-multus-certs\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799088    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-sysctl-conf\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799124    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-cnibin\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799153    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799184    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-socket-dir-parent\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-conf-dir\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fbr4\" (UniqueName: \"kubernetes.io/projected/5643d33f-7080-4c6e-b5bf-bd7104cbae98-kube-api-access-6fbr4\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799310    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-modprobe-d\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799391    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799398    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-lib-modules\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799435    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8406d9dc-3cba-47f7-9448-4ebc5d75972f-cni-binary-copy\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799466    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-run-netns\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799491    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-daemon-config\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799518    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmnm7\" (UniqueName: \"kubernetes.io/projected/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-kube-api-access-wmnm7\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799521    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:14.800404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799570    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gm4b\" (UniqueName: \"kubernetes.io/projected/8406d9dc-3cba-47f7-9448-4ebc5d75972f-kube-api-access-6gm4b\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcf2j\" (UniqueName: \"kubernetes.io/projected/e7314837-02ee-4776-9fcf-430ad544afd9-kube-api-access-zcf2j\") pod \"iptables-alerter-dqq42\" (UID: \"e7314837-02ee-4776-9fcf-430ad544afd9\") " pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799657    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-sysconfig\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-kubernetes\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799704    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-var-lib-kubelet\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-tmp\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkdbv\" (UniqueName: \"kubernetes.io/projected/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-kube-api-access-kkdbv\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8406d9dc-3cba-47f7-9448-4ebc5d75972f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799763    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799793    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-sys\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-os-release\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799832    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a35786b-a78a-4291-ba65-907e78441e0c-agent-certs\") pod \"konnectivity-agent-rx4xb\" (UID: \"8a35786b-a78a-4291-ba65-907e78441e0c\") " pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799847    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a35786b-a78a-4291-ba65-907e78441e0c-konnectivity-ca\") pod \"konnectivity-agent-rx4xb\" (UID: \"8a35786b-a78a-4291-ba65-907e78441e0c\") " pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799879    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7db173ed-c34e-43b4-9251-835908a35f70-dbus\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-cni-dir\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799918    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-cnibin\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-var-lib-kubelet\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.801749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.799981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7db173ed-c34e-43b4-9251-835908a35f70-kubelet-config\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:14.802824 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.800011    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-cni-binary-copy\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.802824 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.800687    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kwgrs\""
Jul 30 16:36:14.802824 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.800726    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 16:36:14.802824 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.800889    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4c6mb\""
Jul 30 16:36:14.810166 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.810138    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 16:36:14.831753 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.831724    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wnm9v"
Jul 30 16:36:14.840793 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.840770    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wnm9v"
Jul 30 16:36:14.890335 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.890304    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 16:36:14.901087 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901049    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmnm7\" (UniqueName: \"kubernetes.io/projected/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-kube-api-access-wmnm7\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.901282 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7314837-02ee-4776-9fcf-430ad544afd9-host-slash\") pod \"iptables-alerter-dqq42\" (UID: \"e7314837-02ee-4776-9fcf-430ad544afd9\") " pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.901282 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-os-release\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.901282 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901167    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-run-ovn\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.901282 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901191    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-cni-netd\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.901282 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901211    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7314837-02ee-4776-9fcf-430ad544afd9-host-slash\") pod \"iptables-alerter-dqq42\" (UID: \"e7314837-02ee-4776-9fcf-430ad544afd9\") " pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.901282 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901220    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fbr4\" (UniqueName: \"kubernetes.io/projected/5643d33f-7080-4c6e-b5bf-bd7104cbae98-kube-api-access-6fbr4\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:14.901282 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-systemd\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901299    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-os-release\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901352    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-run-multus-certs\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-device-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901417    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-run-multus-certs\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901434    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-systemd\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901460    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0c926555-bc21-4e17-98de-15f8d25b951a-serviceca\") pod \"node-ca-kw2nz\" (UID: \"0c926555-bc21-4e17-98de-15f8d25b951a\") " pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901477    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901495    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-sysctl-conf\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-cnibin\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901544    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901566    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-socket-dir-parent\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901594    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-conf-dir\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901615    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qds6b\" (UniqueName: \"kubernetes.io/projected/03a2ede1-e0f7-413e-9c06-a17f08d59575-kube-api-access-qds6b\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901661    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-ovn-node-metrics-cert\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901664    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-cnibin\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.901734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901686    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8406d9dc-3cba-47f7-9448-4ebc5d75972f-cni-binary-copy\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901699    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-sysctl-conf\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-daemon-config\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901758    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-socket-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901784    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-slash\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901815    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gm4b\" (UniqueName: \"kubernetes.io/projected/8406d9dc-3cba-47f7-9448-4ebc5d75972f-kube-api-access-6gm4b\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-socket-dir-parent\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-tmp\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901872    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a35786b-a78a-4291-ba65-907e78441e0c-agent-certs\") pod \"konnectivity-agent-rx4xb\" (UID: \"8a35786b-a78a-4291-ba65-907e78441e0c\") " pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901887    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-conf-dir\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-hostroot\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901925    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-registration-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.901953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-systemd-units\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902002    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-run-systemd\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902054    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-run-openvswitch\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-sys\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902113    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-os-release\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.902532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902139    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7db173ed-c34e-43b4-9251-835908a35f70-dbus\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-cni-dir\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902209    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-cnibin\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-sys\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-etc-openvswitch\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902278    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902306    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4z5g\" (UniqueName: \"kubernetes.io/projected/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-kube-api-access-z4z5g\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902313    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-daemon-config\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902331    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-kubelet\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902357    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-run-netns\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902394    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-node-log\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-hostroot\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c926555-bc21-4e17-98de-15f8d25b951a-host\") pod \"node-ca-kw2nz\" (UID: \"0c926555-bc21-4e17-98de-15f8d25b951a\") " pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902427    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902442    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7db173ed-c34e-43b4-9251-835908a35f70-dbus\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902458    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7314837-02ee-4776-9fcf-430ad544afd9-iptables-alerter-script\") pod \"iptables-alerter-dqq42\" (UID: \"e7314837-02ee-4776-9fcf-430ad544afd9\") " pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902711    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-multus-cni-dir\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902780    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-cnibin\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.903352 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-os-release\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902832    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-system-cni-dir\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902856    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8406d9dc-3cba-47f7-9448-4ebc5d75972f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp2xf\" (UniqueName: \"kubernetes.io/projected/264e7227-dd74-4d87-9f65-107983fb8538-kube-api-access-tp2xf\") pod \"node-resolver-79c74\" (UID: \"264e7227-dd74-4d87-9f65-107983fb8538\") " pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902897    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-sysctl-d\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-run\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902937    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-host\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902951    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8406d9dc-3cba-47f7-9448-4ebc5d75972f-system-cni-dir\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903006    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8406d9dc-3cba-47f7-9448-4ebc5d75972f-cni-binary-copy\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.903025    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903033    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-sysctl-d\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903062    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7314837-02ee-4776-9fcf-430ad544afd9-iptables-alerter-script\") pod \"iptables-alerter-dqq42\" (UID: \"e7314837-02ee-4776-9fcf-430ad544afd9\") " pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.902958    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.903081    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret podName:7db173ed-c34e-43b4-9251-835908a35f70 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:15.403061792 +0000 UTC m=+3.005489402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret") pod "global-pull-secret-syncer-bwspl" (UID: "7db173ed-c34e-43b4-9251-835908a35f70") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903113    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-host\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903068    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-run\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903116    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/264e7227-dd74-4d87-9f65-107983fb8538-tmp-dir\") pod \"node-resolver-79c74\" (UID: \"264e7227-dd74-4d87-9f65-107983fb8538\") " pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.904255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-var-lib-cni-bin\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-var-lib-cni-multus\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903232    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-var-lib-cni-bin\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-var-lib-cni-multus\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/264e7227-dd74-4d87-9f65-107983fb8538-hosts-file\") pod \"node-resolver-79c74\" (UID: \"264e7227-dd74-4d87-9f65-107983fb8538\") " pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903334    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-system-cni-dir\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903344    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/264e7227-dd74-4d87-9f65-107983fb8538-hosts-file\") pod \"node-resolver-79c74\" (UID: \"264e7227-dd74-4d87-9f65-107983fb8538\") " pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903357    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-run-k8s-cni-cncf-io\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/264e7227-dd74-4d87-9f65-107983fb8538-tmp-dir\") pod \"node-resolver-79c74\" (UID: \"264e7227-dd74-4d87-9f65-107983fb8538\") " pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-sys-fs\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903394    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-system-cni-dir\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903403    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-etc-selinux\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903435    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903438    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-run-k8s-cni-cncf-io\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-env-overrides\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-modprobe-d\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.905080 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903502    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-lib-modules\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903516    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8406d9dc-3cba-47f7-9448-4ebc5d75972f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903557    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-run-netns\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903586    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-lib-modules\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903604    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-run-netns\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903597    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-modprobe-d\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903619    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcf2j\" (UniqueName: \"kubernetes.io/projected/e7314837-02ee-4776-9fcf-430ad544afd9-kube-api-access-zcf2j\") pod \"iptables-alerter-dqq42\" (UID: \"e7314837-02ee-4776-9fcf-430ad544afd9\") " pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903724    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-sysconfig\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.903734    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903754    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-kubernetes\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.903787    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs podName:5643d33f-7080-4c6e-b5bf-bd7104cbae98 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:15.403769514 +0000 UTC m=+3.006197125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs") pod "network-metrics-daemon-qzc6g" (UID: "5643d33f-7080-4c6e-b5bf-bd7104cbae98") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903806    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-var-lib-kubelet\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903830    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-kubernetes\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkdbv\" (UniqueName: \"kubernetes.io/projected/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-kube-api-access-kkdbv\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903877    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8406d9dc-3cba-47f7-9448-4ebc5d75972f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903909    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a35786b-a78a-4291-ba65-907e78441e0c-konnectivity-ca\") pod \"konnectivity-agent-rx4xb\" (UID: \"8a35786b-a78a-4291-ba65-907e78441e0c\") " pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:14.905852 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903922    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-sysconfig\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.903934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-etc-kubernetes\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-var-lib-kubelet\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904113    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-var-lib-kubelet\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904146    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-var-lib-openvswitch\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-log-socket\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-cni-bin\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904222    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-ovnkube-config\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-ovnkube-script-lib\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904297    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-host-var-lib-kubelet\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904308    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7db173ed-c34e-43b4-9251-835908a35f70-kubelet-config\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904342    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-cni-binary-copy\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lnqf\" (UniqueName: \"kubernetes.io/projected/0c926555-bc21-4e17-98de-15f8d25b951a-kube-api-access-8lnqf\") pod \"node-ca-kw2nz\" (UID: \"0c926555-bc21-4e17-98de-15f8d25b951a\") " pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904379    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7db173ed-c34e-43b4-9251-835908a35f70-kubelet-config\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-tuned\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904429    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8406d9dc-3cba-47f7-9448-4ebc5d75972f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904478    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-etc-kubernetes\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.906711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904522    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a35786b-a78a-4291-ba65-907e78441e0c-konnectivity-ca\") pod \"konnectivity-agent-rx4xb\" (UID: \"8a35786b-a78a-4291-ba65-907e78441e0c\") " pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:14.907273 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.904880    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-cni-binary-copy\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.907273 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.906353    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-tmp\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.907273 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.906557    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a35786b-a78a-4291-ba65-907e78441e0c-agent-certs\") pod \"konnectivity-agent-rx4xb\" (UID: \"8a35786b-a78a-4291-ba65-907e78441e0c\") " pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:14.907273 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.906698    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-etc-tuned\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.908746 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.908727    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:14.908746 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.908749    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:14.908907 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.908759    2569 projected.go:196] Error preparing data for projected volume kube-api-access-jzpj5 for pod openshift-network-diagnostics/network-check-target-vd7hz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:14.908907 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:14.908829    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5 podName:046e7ae1-377f-4cd0-beba-28b76b5ceeac nodeName:}" failed. No retries permitted until 2026-07-30 16:36:15.408792272 +0000 UTC m=+3.011219894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jzpj5" (UniqueName: "kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5") pod "network-check-target-vd7hz" (UID: "046e7ae1-377f-4cd0-beba-28b76b5ceeac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:14.911454 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.911426    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fbr4\" (UniqueName: \"kubernetes.io/projected/5643d33f-7080-4c6e-b5bf-bd7104cbae98-kube-api-access-6fbr4\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:14.911574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.911513    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmnm7\" (UniqueName: \"kubernetes.io/projected/34b4ec6e-1d65-4042-9f8a-7cd4ff00b336-kube-api-access-wmnm7\") pod \"multus-dmktj\" (UID: \"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336\") " pod="openshift-multus/multus-dmktj"
Jul 30 16:36:14.911671 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.911570    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gm4b\" (UniqueName: \"kubernetes.io/projected/8406d9dc-3cba-47f7-9448-4ebc5d75972f-kube-api-access-6gm4b\") pod \"multus-additional-cni-plugins-ngd2j\" (UID: \"8406d9dc-3cba-47f7-9448-4ebc5d75972f\") " pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:14.912867 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.912845    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp2xf\" (UniqueName: \"kubernetes.io/projected/264e7227-dd74-4d87-9f65-107983fb8538-kube-api-access-tp2xf\") pod \"node-resolver-79c74\" (UID: \"264e7227-dd74-4d87-9f65-107983fb8538\") " pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:14.916975 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.916956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcf2j\" (UniqueName: \"kubernetes.io/projected/e7314837-02ee-4776-9fcf-430ad544afd9-kube-api-access-zcf2j\") pod \"iptables-alerter-dqq42\" (UID: \"e7314837-02ee-4776-9fcf-430ad544afd9\") " pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:14.923261 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.923238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkdbv\" (UniqueName: \"kubernetes.io/projected/9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3-kube-api-access-kkdbv\") pod \"tuned-b6c2j\" (UID: \"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3\") " pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:14.950001 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:14.949965    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 16:36:15.001203 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.001140    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal" event={"ID":"47a748d50900f64c692e07ae4042b0bd","Type":"ContainerStarted","Data":"5f9f5448f8e1744adea862d98b3aae8eabd9821d0d9e7838fd10061aaa93ed8b"}
Jul 30 16:36:15.002393 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.002359    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal" event={"ID":"3c77bfff64908136b53abdd3eaff108d","Type":"ContainerStarted","Data":"005f3edd6478964e3c8f3a612edc713cfbbbcd760ad38544a3dda5821f8a5fe7"}
Jul 30 16:36:15.004688 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004659    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-run-ovn\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.004817 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004697    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-cni-netd\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.004817 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004717    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-device-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.004817 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004735    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0c926555-bc21-4e17-98de-15f8d25b951a-serviceca\") pod \"node-ca-kw2nz\" (UID: \"0c926555-bc21-4e17-98de-15f8d25b951a\") " pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:15.004817 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004777    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qds6b\" (UniqueName: \"kubernetes.io/projected/03a2ede1-e0f7-413e-9c06-a17f08d59575-kube-api-access-qds6b\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.004817 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004788    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-cni-netd\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.004817 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004802    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-ovn-node-metrics-cert\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005061 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004828    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-socket-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.005061 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004845    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-device-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.005061 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004850    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-slash\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005061 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004879    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-slash\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005061 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004910    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-registration-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.005061 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004937    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-systemd-units\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005061 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004963    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-run-systemd\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005061 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005010    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-run-openvswitch\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005061 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005044    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-etc-openvswitch\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005070    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4z5g\" (UniqueName: \"kubernetes.io/projected/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-kube-api-access-z4z5g\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005122    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-kubelet\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-run-netns\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005173    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-node-log\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c926555-bc21-4e17-98de-15f8d25b951a-host\") pod \"node-ca-kw2nz\" (UID: \"0c926555-bc21-4e17-98de-15f8d25b951a\") " pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005249    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-sys-fs\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005288    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0c926555-bc21-4e17-98de-15f8d25b951a-serviceca\") pod \"node-ca-kw2nz\" (UID: \"0c926555-bc21-4e17-98de-15f8d25b951a\") " pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005305    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-etc-selinux\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.004790    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-run-ovn\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005331    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005358    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-env-overrides\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005411    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-var-lib-openvswitch\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005434    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-log-socket\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005488    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-log-socket\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005542    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-run-netns\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005567    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005576    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-run-systemd\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005564    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-kubelet\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005598    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-run-openvswitch\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005632    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-etc-openvswitch\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005636    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-node-log\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c926555-bc21-4e17-98de-15f8d25b951a-host\") pod \"node-ca-kw2nz\" (UID: \"0c926555-bc21-4e17-98de-15f8d25b951a\") " pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005679    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-systemd-units\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005707    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-registration-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005710    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-sys-fs\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005725    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.005822 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005758    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005792    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-cni-bin\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005821    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-ovnkube-config\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005839    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-var-lib-openvswitch\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-ovnkube-script-lib\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005876    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lnqf\" (UniqueName: \"kubernetes.io/projected/0c926555-bc21-4e17-98de-15f8d25b951a-kube-api-access-8lnqf\") pod \"node-ca-kw2nz\" (UID: \"0c926555-bc21-4e17-98de-15f8d25b951a\") " pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005914    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-host-cni-bin\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.005847    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-etc-selinux\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.006119    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/03a2ede1-e0f7-413e-9c06-a17f08d59575-socket-dir\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.006328    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-env-overrides\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.006609    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-ovnkube-config\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.006702 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.006615    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-ovnkube-script-lib\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.008701 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.008672    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-ovn-node-metrics-cert\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.015075 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.015044    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qds6b\" (UniqueName: \"kubernetes.io/projected/03a2ede1-e0f7-413e-9c06-a17f08d59575-kube-api-access-qds6b\") pod \"aws-ebs-csi-driver-node-9c2tg\" (UID: \"03a2ede1-e0f7-413e-9c06-a17f08d59575\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.015215 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.015174    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4z5g\" (UniqueName: \"kubernetes.io/projected/7804a0c3-7c36-42a8-aadf-ff4464c3cdbe-kube-api-access-z4z5g\") pod \"ovnkube-node-44szc\" (UID: \"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.015413 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.015388    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lnqf\" (UniqueName: \"kubernetes.io/projected/0c926555-bc21-4e17-98de-15f8d25b951a-kube-api-access-8lnqf\") pod \"node-ca-kw2nz\" (UID: \"0c926555-bc21-4e17-98de-15f8d25b951a\") " pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:15.092521 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.092467    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dqq42"
Jul 30 16:36:15.101897 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.101864    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b6c2j"
Jul 30 16:36:15.103098 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:15.103065    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7314837_02ee_4776_9fcf_430ad544afd9.slice/crio-5baa6e36f517af8386f465a2057a905a6297d8cca8b9994d429076ee76834370 WatchSource:0}: Error finding container 5baa6e36f517af8386f465a2057a905a6297d8cca8b9994d429076ee76834370: Status 404 returned error can't find the container with id 5baa6e36f517af8386f465a2057a905a6297d8cca8b9994d429076ee76834370
Jul 30 16:36:15.113834 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.113622    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-79c74"
Jul 30 16:36:15.118384 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.118362    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ngd2j"
Jul 30 16:36:15.124785 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:15.124743    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264e7227_dd74_4d87_9f65_107983fb8538.slice/crio-c44ff63bc4943ee2a9470192f9dde9b6e6cf4dc8cc10b2313c2e251c106747ef WatchSource:0}: Error finding container c44ff63bc4943ee2a9470192f9dde9b6e6cf4dc8cc10b2313c2e251c106747ef: Status 404 returned error can't find the container with id c44ff63bc4943ee2a9470192f9dde9b6e6cf4dc8cc10b2313c2e251c106747ef
Jul 30 16:36:15.126757 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.126735    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dmktj"
Jul 30 16:36:15.135669 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.134410    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:15.139025 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:15.138963    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8406d9dc_3cba_47f7_9448_4ebc5d75972f.slice/crio-6dfdfd91dc19f7b6e281fd6183b8c13f49e237434b4fdfbf4c2da0f22275340a WatchSource:0}: Error finding container 6dfdfd91dc19f7b6e281fd6183b8c13f49e237434b4fdfbf4c2da0f22275340a: Status 404 returned error can't find the container with id 6dfdfd91dc19f7b6e281fd6183b8c13f49e237434b4fdfbf4c2da0f22275340a
Jul 30 16:36:15.141412 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.141389    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg"
Jul 30 16:36:15.148542 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.148451    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:15.149851 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:15.149803    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a35786b_a78a_4291_ba65_907e78441e0c.slice/crio-cbe844041f07f97d651620219c4d0ac7e25c6a3ea9dd0c41377a475ee744d86a WatchSource:0}: Error finding container cbe844041f07f97d651620219c4d0ac7e25c6a3ea9dd0c41377a475ee744d86a: Status 404 returned error can't find the container with id cbe844041f07f97d651620219c4d0ac7e25c6a3ea9dd0c41377a475ee744d86a
Jul 30 16:36:15.152683 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:15.152424    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b4ec6e_1d65_4042_9f8a_7cd4ff00b336.slice/crio-bec67b7ab8ab571f82dd03ed8c2e5accd91d73125429fa3881026e0885a177ea WatchSource:0}: Error finding container bec67b7ab8ab571f82dd03ed8c2e5accd91d73125429fa3881026e0885a177ea: Status 404 returned error can't find the container with id bec67b7ab8ab571f82dd03ed8c2e5accd91d73125429fa3881026e0885a177ea
Jul 30 16:36:15.154838 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.154194    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kw2nz"
Jul 30 16:36:15.159288 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:15.158946    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a2ede1_e0f7_413e_9c06_a17f08d59575.slice/crio-66aa17f68b645f77ed6c5f52fc95d3e657abed6f145e686f02ae11072485229b WatchSource:0}: Error finding container 66aa17f68b645f77ed6c5f52fc95d3e657abed6f145e686f02ae11072485229b: Status 404 returned error can't find the container with id 66aa17f68b645f77ed6c5f52fc95d3e657abed6f145e686f02ae11072485229b
Jul 30 16:36:15.161857 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:15.161823    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7804a0c3_7c36_42a8_aadf_ff4464c3cdbe.slice/crio-4e5918febfbd667e42dc65f711304b869e7cd5e817907005d18343e334b7565f WatchSource:0}: Error finding container 4e5918febfbd667e42dc65f711304b869e7cd5e817907005d18343e334b7565f: Status 404 returned error can't find the container with id 4e5918febfbd667e42dc65f711304b869e7cd5e817907005d18343e334b7565f
Jul 30 16:36:15.169739 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:15.169707    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c926555_bc21_4e17_98de_15f8d25b951a.slice/crio-2792fefc66cb00c1f1c9440ad462980f785ffdb7d4a1b6bc592645ae8733c10d WatchSource:0}: Error finding container 2792fefc66cb00c1f1c9440ad462980f785ffdb7d4a1b6bc592645ae8733c10d: Status 404 returned error can't find the container with id 2792fefc66cb00c1f1c9440ad462980f785ffdb7d4a1b6bc592645ae8733c10d
Jul 30 16:36:15.409032 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.408934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:15.409032 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.409007    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:15.409251 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.409071    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:15.409251 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:15.409150    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:15.409251 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:15.409178    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:15.409251 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:15.409185    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:15.409251 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:15.409193    2569 projected.go:196] Error preparing data for projected volume kube-api-access-jzpj5 for pod openshift-network-diagnostics/network-check-target-vd7hz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:15.409251 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:15.409245    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs podName:5643d33f-7080-4c6e-b5bf-bd7104cbae98 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:16.40922094 +0000 UTC m=+4.011648546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs") pod "network-metrics-daemon-qzc6g" (UID: "5643d33f-7080-4c6e-b5bf-bd7104cbae98") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:15.409662 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:15.409266    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5 podName:046e7ae1-377f-4cd0-beba-28b76b5ceeac nodeName:}" failed. No retries permitted until 2026-07-30 16:36:16.409255203 +0000 UTC m=+4.011682813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzpj5" (UniqueName: "kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5") pod "network-check-target-vd7hz" (UID: "046e7ae1-377f-4cd0-beba-28b76b5ceeac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:15.409662 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:15.409296    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:15.409662 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:15.409337    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret podName:7db173ed-c34e-43b4-9251-835908a35f70 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:16.409319829 +0000 UTC m=+4.011747444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret") pod "global-pull-secret-syncer-bwspl" (UID: "7db173ed-c34e-43b4-9251-835908a35f70") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:15.843544 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.843391    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 16:31:15 +0000 UTC" deadline="2028-02-21 01:26:48.580231789 +0000 UTC"
Jul 30 16:36:15.843544 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.843454    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13688h50m32.736785004s"
Jul 30 16:36:15.994524 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:15.994466    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:15.994754 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:15.994617    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:16.009760 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.009708    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dmktj" event={"ID":"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336","Type":"ContainerStarted","Data":"bec67b7ab8ab571f82dd03ed8c2e5accd91d73125429fa3881026e0885a177ea"}
Jul 30 16:36:16.022904 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.022860    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rx4xb" event={"ID":"8a35786b-a78a-4291-ba65-907e78441e0c","Type":"ContainerStarted","Data":"cbe844041f07f97d651620219c4d0ac7e25c6a3ea9dd0c41377a475ee744d86a"}
Jul 30 16:36:16.024712 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.024681    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-79c74" event={"ID":"264e7227-dd74-4d87-9f65-107983fb8538","Type":"ContainerStarted","Data":"c44ff63bc4943ee2a9470192f9dde9b6e6cf4dc8cc10b2313c2e251c106747ef"}
Jul 30 16:36:16.032234 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.032194    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dqq42" event={"ID":"e7314837-02ee-4776-9fcf-430ad544afd9","Type":"ContainerStarted","Data":"5baa6e36f517af8386f465a2057a905a6297d8cca8b9994d429076ee76834370"}
Jul 30 16:36:16.037846 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.037804    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg" event={"ID":"03a2ede1-e0f7-413e-9c06-a17f08d59575","Type":"ContainerStarted","Data":"66aa17f68b645f77ed6c5f52fc95d3e657abed6f145e686f02ae11072485229b"}
Jul 30 16:36:16.039629 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.039597    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngd2j" event={"ID":"8406d9dc-3cba-47f7-9448-4ebc5d75972f","Type":"ContainerStarted","Data":"6dfdfd91dc19f7b6e281fd6183b8c13f49e237434b4fdfbf4c2da0f22275340a"}
Jul 30 16:36:16.041780 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.041749    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b6c2j" event={"ID":"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3","Type":"ContainerStarted","Data":"95b9cbe5e1514d864dc597d3f0a4272196108be412ce1454d435e7f34c1e626b"}
Jul 30 16:36:16.044501 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.044438    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kw2nz" event={"ID":"0c926555-bc21-4e17-98de-15f8d25b951a","Type":"ContainerStarted","Data":"2792fefc66cb00c1f1c9440ad462980f785ffdb7d4a1b6bc592645ae8733c10d"}
Jul 30 16:36:16.047218 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.047188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" event={"ID":"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe","Type":"ContainerStarted","Data":"4e5918febfbd667e42dc65f711304b869e7cd5e817907005d18343e334b7565f"}
Jul 30 16:36:16.417793 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.417743    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:16.417995 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.417863    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:16.417995 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.417903    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:16.418116 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.418060    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:16.418169 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.418138    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs podName:5643d33f-7080-4c6e-b5bf-bd7104cbae98 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:18.418117406 +0000 UTC m=+6.020545015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs") pod "network-metrics-daemon-qzc6g" (UID: "5643d33f-7080-4c6e-b5bf-bd7104cbae98") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:16.418969 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.418611    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:16.418969 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.418639    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:16.418969 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.418653    2569 projected.go:196] Error preparing data for projected volume kube-api-access-jzpj5 for pod openshift-network-diagnostics/network-check-target-vd7hz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:16.418969 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.418795    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:16.419429 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.419337    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5 podName:046e7ae1-377f-4cd0-beba-28b76b5ceeac nodeName:}" failed. No retries permitted until 2026-07-30 16:36:18.419311131 +0000 UTC m=+6.021738753 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzpj5" (UniqueName: "kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5") pod "network-check-target-vd7hz" (UID: "046e7ae1-377f-4cd0-beba-28b76b5ceeac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:16.419429 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.419375    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret podName:7db173ed-c34e-43b4-9251-835908a35f70 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:18.419363395 +0000 UTC m=+6.021791014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret") pod "global-pull-secret-syncer-bwspl" (UID: "7db173ed-c34e-43b4-9251-835908a35f70") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:16.844574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.844420    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 16:31:15 +0000 UTC" deadline="2028-02-16 08:58:54.505716936 +0000 UTC"
Jul 30 16:36:16.844574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.844472    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13576h22m37.661249603s"
Jul 30 16:36:16.997571 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.996818    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:16.997571 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.996957    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:16.997571 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:16.997409    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:16.997571 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:16.997521    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:17.994533 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:17.994490    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:17.995173 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:17.994668    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:18.436826 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:18.436729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:18.436826 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:18.436801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:18.437069 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:18.436857    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:18.437123 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.437103    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:18.437224 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.437130    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:18.437224 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.437145    2569 projected.go:196] Error preparing data for projected volume kube-api-access-jzpj5 for pod openshift-network-diagnostics/network-check-target-vd7hz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:18.437224 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.437213    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5 podName:046e7ae1-377f-4cd0-beba-28b76b5ceeac nodeName:}" failed. No retries permitted until 2026-07-30 16:36:22.437190304 +0000 UTC m=+10.039617934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzpj5" (UniqueName: "kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5") pod "network-check-target-vd7hz" (UID: "046e7ae1-377f-4cd0-beba-28b76b5ceeac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:18.437661 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.437639    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:18.437771 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.437721    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret podName:7db173ed-c34e-43b4-9251-835908a35f70 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:22.437701384 +0000 UTC m=+10.040129010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret") pod "global-pull-secret-syncer-bwspl" (UID: "7db173ed-c34e-43b4-9251-835908a35f70") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:18.437771 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.437789    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:18.437944 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.437837    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs podName:5643d33f-7080-4c6e-b5bf-bd7104cbae98 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:22.437818886 +0000 UTC m=+10.040246510 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs") pod "network-metrics-daemon-qzc6g" (UID: "5643d33f-7080-4c6e-b5bf-bd7104cbae98") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:18.997127 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:18.996574    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:18.997127 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.996750    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:18.997926 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:18.997779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:18.997926 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:18.997887    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:19.995346 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:19.995308    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:19.995592 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:19.995465    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:20.994833 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:20.994750    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:20.995287 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:20.994885    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:20.995287 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:20.994981    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:20.995287 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:20.995110    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:21.995328 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:21.995238    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:21.995813 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:21.995377    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:22.474266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:22.474336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:22.474385    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.474538    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.474559    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.474572    2569 projected.go:196] Error preparing data for projected volume kube-api-access-jzpj5 for pod openshift-network-diagnostics/network-check-target-vd7hz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.474636    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5 podName:046e7ae1-377f-4cd0-beba-28b76b5ceeac nodeName:}" failed. No retries permitted until 2026-07-30 16:36:30.474615359 +0000 UTC m=+18.077042969 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzpj5" (UniqueName: "kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5") pod "network-check-target-vd7hz" (UID: "046e7ae1-377f-4cd0-beba-28b76b5ceeac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.474634    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.474711    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret podName:7db173ed-c34e-43b4-9251-835908a35f70 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:30.474691269 +0000 UTC m=+18.077118892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret") pod "global-pull-secret-syncer-bwspl" (UID: "7db173ed-c34e-43b4-9251-835908a35f70") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.474780    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:22.475199 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.474817    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs podName:5643d33f-7080-4c6e-b5bf-bd7104cbae98 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:30.474804355 +0000 UTC m=+18.077231962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs") pod "network-metrics-daemon-qzc6g" (UID: "5643d33f-7080-4c6e-b5bf-bd7104cbae98") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:22.997583 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:22.997103    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:22.997583 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.997222    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:22.997583 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:22.997472    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:22.997583 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:22.997572    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:23.994815 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:23.994737    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:23.994988 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:23.994869    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:24.994814 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:24.994784    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:24.995226 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:24.994931    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:24.995226 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:24.994963    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:24.995226 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:24.995085    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:25.995511 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:25.995479    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:25.995883 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:25.995574    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:26.995575 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:26.995536    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:26.995575 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:26.995579    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:26.996005 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:26.995673    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:26.996005 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:26.995835    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:27.995113 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:27.995073    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:27.995297 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:27.995205    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:28.994933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:28.994893    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:28.995384 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:28.994943    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:28.995384 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:28.995025    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:28.995384 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:28.995142    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:29.995241 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:29.995204    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:29.995743 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:29.995339    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:30.534827 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:30.534784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:30.535016 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:30.534842    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:30.535016 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:30.534879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:30.535016 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.534951    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:30.535166 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.535010    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:30.535166 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.535027    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs podName:5643d33f-7080-4c6e-b5bf-bd7104cbae98 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:46.535009471 +0000 UTC m=+34.137437095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs") pod "network-metrics-daemon-qzc6g" (UID: "5643d33f-7080-4c6e-b5bf-bd7104cbae98") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:30.535166 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.535033    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:30.535166 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.535054    2569 projected.go:196] Error preparing data for projected volume kube-api-access-jzpj5 for pod openshift-network-diagnostics/network-check-target-vd7hz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:30.535166 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.534951    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:30.535166 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.535115    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret podName:7db173ed-c34e-43b4-9251-835908a35f70 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:46.535095806 +0000 UTC m=+34.137523412 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret") pod "global-pull-secret-syncer-bwspl" (UID: "7db173ed-c34e-43b4-9251-835908a35f70") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:30.535166 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.535133    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5 podName:046e7ae1-377f-4cd0-beba-28b76b5ceeac nodeName:}" failed. No retries permitted until 2026-07-30 16:36:46.535124036 +0000 UTC m=+34.137551645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzpj5" (UniqueName: "kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5") pod "network-check-target-vd7hz" (UID: "046e7ae1-377f-4cd0-beba-28b76b5ceeac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:30.995532 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:30.995446    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:30.995976 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:30.995445    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:30.995976 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.995600    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:30.995976 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:30.995683    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:31.995441 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:31.995402    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:31.995645 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:31.995519    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:32.995623 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:32.995575    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:32.995800 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:32.995735    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:32.995800 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:32.995789    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:32.996096 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:32.995905    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:33.995070 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:33.994998    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:33.995188 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:33.995108    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:34.994997 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:34.994703    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:34.995680 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:34.994709    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:34.995680 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:34.995501    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:34.995680 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:34.995656    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:35.088329 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.088252    2569 generic.go:356] "Generic (PLEG): container finished" podID="8406d9dc-3cba-47f7-9448-4ebc5d75972f" containerID="987a07fdc992c2998110fcb45359793b78e479e3fea62f6667071a77a0266934" exitCode=0
Jul 30 16:36:35.088467 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.088339    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngd2j" event={"ID":"8406d9dc-3cba-47f7-9448-4ebc5d75972f","Type":"ContainerDied","Data":"987a07fdc992c2998110fcb45359793b78e479e3fea62f6667071a77a0266934"}
Jul 30 16:36:35.090092 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.090066    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b6c2j" event={"ID":"9eeb5b98-b3ef-4f11-9949-cc23ac7b10b3","Type":"ContainerStarted","Data":"0f4ba3b6bbb9ce32d28e0523a7089ab687ee36017a17b7f4349a289d0eece236"}
Jul 30 16:36:35.091855 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.091833    2569 generic.go:356] "Generic (PLEG): container finished" podID="47a748d50900f64c692e07ae4042b0bd" containerID="64cfb7fce8bd5a419823869c14e680154fc0f3de0a5979fb07d6474b53de9f5d" exitCode=0
Jul 30 16:36:35.091973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.091917    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal" event={"ID":"47a748d50900f64c692e07ae4042b0bd","Type":"ContainerDied","Data":"64cfb7fce8bd5a419823869c14e680154fc0f3de0a5979fb07d6474b53de9f5d"}
Jul 30 16:36:35.093497 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.093473    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kw2nz" event={"ID":"0c926555-bc21-4e17-98de-15f8d25b951a","Type":"ContainerStarted","Data":"6d5c4511170f93f47fd591aff0af06b679718c42859b28f354cb69bf7c48daff"}
Jul 30 16:36:35.096694 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.096670    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" event={"ID":"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe","Type":"ContainerStarted","Data":"a17c18fd3e5f07e81be3fdace531c8e475b7d18402d0cf5adf403c201d23cee5"}
Jul 30 16:36:35.096785 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.096742    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" event={"ID":"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe","Type":"ContainerStarted","Data":"d18ba52cf7bac98d798164269ffdd57fe63d5728fff3ceb4d15b381c93399033"}
Jul 30 16:36:35.096785 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.096784    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" event={"ID":"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe","Type":"ContainerStarted","Data":"b2c932d72cd9f50cc79646bc616c38abdf933ff73ded79f5c89dc48057e1982f"}
Jul 30 16:36:35.096893 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.096823    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" event={"ID":"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe","Type":"ContainerStarted","Data":"fcb708508810be91c91566a8e2ee30d601b16cc024b7c9be9fde9d4d031f2cf9"}
Jul 30 16:36:35.096893 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.096866    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" event={"ID":"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe","Type":"ContainerStarted","Data":"967ca604bfa67c247ca6c656b42f407a61f5d88b2dbbfbd33939bb3dc468bcf8"}
Jul 30 16:36:35.096978 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.096909    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" event={"ID":"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe","Type":"ContainerStarted","Data":"f466bcfd832df478ffccb6bc6bd4e55d7c33094a3225d3e8052c1127b30ac869"}
Jul 30 16:36:35.098248 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.098221    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dmktj" event={"ID":"34b4ec6e-1d65-4042-9f8a-7cd4ff00b336","Type":"ContainerStarted","Data":"29d98bc39c93aa9c6b85d198057aba94a6a0869c06945ee5be08484030f9a28a"}
Jul 30 16:36:35.099656 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.099638    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rx4xb" event={"ID":"8a35786b-a78a-4291-ba65-907e78441e0c","Type":"ContainerStarted","Data":"707d0f80cff192054d85f7f477fa5bc6d5e346c8b869b640d4c34a47d46b3a2d"}
Jul 30 16:36:35.101143 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.101125    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-79c74" event={"ID":"264e7227-dd74-4d87-9f65-107983fb8538","Type":"ContainerStarted","Data":"f058d4c4dae53fc3d21560b32d3932a57d58e32733a1acd85f3ba6c80d51a8c1"}
Jul 30 16:36:35.102588 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.102565    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal" event={"ID":"3c77bfff64908136b53abdd3eaff108d","Type":"ContainerStarted","Data":"1857cd1a6631fd01cbfbf5a62f584edb027477e71fa6b22bffc02c992b4b5693"}
Jul 30 16:36:35.104167 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.104143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg" event={"ID":"03a2ede1-e0f7-413e-9c06-a17f08d59575","Type":"ContainerStarted","Data":"1fb726788d5f1d6cb7ce527ae917db30a6bff46d7b275d5f4cc212fe7bcbde76"}
Jul 30 16:36:35.129559 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.129497    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b6c2j" podStartSLOduration=3.10613977 podStartE2EDuration="22.129482817s" totalImagesPullingTime="19.023343047s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:35.12919833 +0000 UTC m=+22.731625956" watchObservedRunningTime="2026-07-30 16:36:35.129482817 +0000 UTC m=+22.731910443"
Jul 30 16:36:35.144780 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.144737    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-79c74" podStartSLOduration=3.621508376 podStartE2EDuration="22.14472233s" totalImagesPullingTime="18.523213954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:35.144051838 +0000 UTC m=+22.746479463" watchObservedRunningTime="2026-07-30 16:36:35.14472233 +0000 UTC m=+22.747149961"
Jul 30 16:36:35.162681 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.162461    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dmktj" podStartSLOduration=2.7469331649999997 podStartE2EDuration="22.162448005s" totalImagesPullingTime="19.41551484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:35.162314743 +0000 UTC m=+22.764742370" watchObservedRunningTime="2026-07-30 16:36:35.162448005 +0000 UTC m=+22.764875631"
Jul 30 16:36:35.195175 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.195131    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rx4xb" podStartSLOduration=3.695210745 podStartE2EDuration="22.195116253s" totalImagesPullingTime="18.499905508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:35.180145623 +0000 UTC m=+22.782573250" watchObservedRunningTime="2026-07-30 16:36:35.195116253 +0000 UTC m=+22.797543878"
Jul 30 16:36:35.195313 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.195295    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kw2nz" podStartSLOduration=3.230528838 podStartE2EDuration="22.195291723s" totalImagesPullingTime="18.964762885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:35.194965199 +0000 UTC m=+22.797392825" watchObservedRunningTime="2026-07-30 16:36:35.195291723 +0000 UTC m=+22.797719380"
Jul 30 16:36:35.225292 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.225247    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-111.ec2.internal" podStartSLOduration=21.225233621 podStartE2EDuration="21.225233621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:35.225006043 +0000 UTC m=+22.827433669" watchObservedRunningTime="2026-07-30 16:36:35.225233621 +0000 UTC m=+22.827661246"
Jul 30 16:36:35.643796 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.643774    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 16:36:35.862194 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.862047    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T16:36:35.643790802Z","UUID":"d14d38c2-1427-4958-9d30-3e678f6c483d","Handler":null,"Name":"","Endpoint":""}
Jul 30 16:36:35.863776 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.863754    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 16:36:35.863917 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.863795    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 16:36:35.995325 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:35.995297    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:35.995834 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:35.995422    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:36.108898 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:36.108863    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg" event={"ID":"03a2ede1-e0f7-413e-9c06-a17f08d59575","Type":"ContainerStarted","Data":"fa9f31ae4e6f9f9ac2a056f1d9c6bce503c518cc819cd96b390ad5b840ccfd30"}
Jul 30 16:36:36.110752 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:36.110722    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal" event={"ID":"47a748d50900f64c692e07ae4042b0bd","Type":"ContainerStarted","Data":"30da5236bae03c71ed480637a33cd4b753fdce2ff70bbbbad41fca12f599e9df"}
Jul 30 16:36:36.112443 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:36.112368    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dqq42" event={"ID":"e7314837-02ee-4776-9fcf-430ad544afd9","Type":"ContainerStarted","Data":"53730b81baf37d21a0476a87791513c7877622ea0de31483499604898dc75ca6"}
Jul 30 16:36:36.130818 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:36.130777    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-111.ec2.internal" podStartSLOduration=22.130764099 podStartE2EDuration="22.130764099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:36.130537258 +0000 UTC m=+23.732964885" watchObservedRunningTime="2026-07-30 16:36:36.130764099 +0000 UTC m=+23.733191724"
Jul 30 16:36:36.151703 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:36.151660    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dqq42" podStartSLOduration=4.174344288 podStartE2EDuration="23.151648078s" totalImagesPullingTime="18.97730379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:36.151444243 +0000 UTC m=+23.753871869" watchObservedRunningTime="2026-07-30 16:36:36.151648078 +0000 UTC m=+23.754075704"
Jul 30 16:36:36.994898 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:36.994872    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:36.994898 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:36.994895    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:36.995145 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:36.994994    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:36.995201 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:36.995131    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:37.994498 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:37.994466    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:37.995040 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:37.994589    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:38.120083 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:38.119868    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg" event={"ID":"03a2ede1-e0f7-413e-9c06-a17f08d59575","Type":"ContainerStarted","Data":"bb89b5f9a5958c1623c460682e0bea06688c6d5a7431fab518b549c10dd4d7b2"}
Jul 30 16:36:38.123672 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:38.123641    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" event={"ID":"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe","Type":"ContainerStarted","Data":"62c2034a26cf0e361a1817144ba1e8ad79d76eafc46fa1a314d59a785223c6e4"}
Jul 30 16:36:38.137688 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:38.137648    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9c2tg" podStartSLOduration=3.722110881 podStartE2EDuration="25.137634623s" totalImagesPullingTime="21.415523742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:38.137482294 +0000 UTC m=+25.739909920" watchObservedRunningTime="2026-07-30 16:36:38.137634623 +0000 UTC m=+25.740062248"
Jul 30 16:36:38.777323 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:38.777284    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:38.777962 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:38.777937    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:38.995143 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:38.995108    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:38.995546 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:38.995251    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:38.995546 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:38.995308    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:38.995546 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:38.995429    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:39.125694 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:39.125597    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:39.126001 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:39.125985    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rx4xb"
Jul 30 16:36:39.995387 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:39.995212    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:39.996075 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:39.995461    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:40.129095 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:40.129066    2569 generic.go:356] "Generic (PLEG): container finished" podID="8406d9dc-3cba-47f7-9448-4ebc5d75972f" containerID="fb3288958a484755a6dddaa297fddf1aacc1d61f4c47282add015cf1ee31e85b" exitCode=0
Jul 30 16:36:40.129276 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:40.129158    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngd2j" event={"ID":"8406d9dc-3cba-47f7-9448-4ebc5d75972f","Type":"ContainerDied","Data":"fb3288958a484755a6dddaa297fddf1aacc1d61f4c47282add015cf1ee31e85b"}
Jul 30 16:36:40.132736 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:40.132713    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" event={"ID":"7804a0c3-7c36-42a8-aadf-ff4464c3cdbe","Type":"ContainerStarted","Data":"22a04de2d837297bab164f8a6912767c0efb85ad23ce6ada4b23692d355baa8c"}
Jul 30 16:36:40.182439 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:40.182389    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-44szc" podStartSLOduration=8.162326578 podStartE2EDuration="27.182376086s" totalImagesPullingTime="19.020049508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:40.181862147 +0000 UTC m=+27.784289782" watchObservedRunningTime="2026-07-30 16:36:40.182376086 +0000 UTC m=+27.784803712"
Jul 30 16:36:40.995863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:40.995547    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:40.996293 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:40.995554    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:40.996293 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:40.995908    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:40.996293 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:40.995995    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:41.137284 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.137251    2569 generic.go:356] "Generic (PLEG): container finished" podID="8406d9dc-3cba-47f7-9448-4ebc5d75972f" containerID="3792305da52910660b77365741e765194f15ffc8826118cf18242b5501279fa5" exitCode=0
Jul 30 16:36:41.137437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.137332    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngd2j" event={"ID":"8406d9dc-3cba-47f7-9448-4ebc5d75972f","Type":"ContainerDied","Data":"3792305da52910660b77365741e765194f15ffc8826118cf18242b5501279fa5"}
Jul 30 16:36:41.137898 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.137875    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:41.138052 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.137960    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:41.138052 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.138008    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:41.153572 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.153548    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:41.153773 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.153733    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:36:41.252905 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.252828    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bwspl"]
Jul 30 16:36:41.253044 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.252954    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:41.253105 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:41.253038    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:41.254435 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.254409    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vd7hz"]
Jul 30 16:36:41.254579 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.254495    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:41.254653 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:41.254575    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:41.254841 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.254821    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qzc6g"]
Jul 30 16:36:41.254915 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:41.254899    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:41.255022 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:41.255005    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:42.144062 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:42.144029    2569 generic.go:356] "Generic (PLEG): container finished" podID="8406d9dc-3cba-47f7-9448-4ebc5d75972f" containerID="348d547780986cb1054c6b069377441fa52a8d4a5f23d5372c154939dc82022c" exitCode=0
Jul 30 16:36:42.144493 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:42.144117    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngd2j" event={"ID":"8406d9dc-3cba-47f7-9448-4ebc5d75972f","Type":"ContainerDied","Data":"348d547780986cb1054c6b069377441fa52a8d4a5f23d5372c154939dc82022c"}
Jul 30 16:36:42.996569 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:42.996535    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:42.996778 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:42.996644    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:42.996778 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:42.996693    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:42.996778 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:42.996695    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:42.996938 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:42.996798    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:42.996938 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:42.996869    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:44.995189 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:44.995141    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:44.995661 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:44.995141    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:44.995661 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:44.995285    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:44.995661 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:44.995141    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:44.995661 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:44.995387    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:44.995661 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:44.995422    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:46.562491 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:46.562229    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:46.562491 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:46.562457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:46.563028 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.562370    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:46.563028 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:46.562508    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:46.563028 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.562553    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret podName:7db173ed-c34e-43b4-9251-835908a35f70 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:18.562536967 +0000 UTC m=+66.164964576 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret") pod "global-pull-secret-syncer-bwspl" (UID: "7db173ed-c34e-43b4-9251-835908a35f70") : object "kube-system"/"original-pull-secret" not registered
Jul 30 16:36:46.563028 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.562578    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:46.563028 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.562648    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs podName:5643d33f-7080-4c6e-b5bf-bd7104cbae98 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:18.562635365 +0000 UTC m=+66.165062969 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs") pod "network-metrics-daemon-qzc6g" (UID: "5643d33f-7080-4c6e-b5bf-bd7104cbae98") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 16:36:46.563028 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.562659    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 16:36:46.563028 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.562677    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 16:36:46.563028 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.562690    2569 projected.go:196] Error preparing data for projected volume kube-api-access-jzpj5 for pod openshift-network-diagnostics/network-check-target-vd7hz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:46.563028 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.562737    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5 podName:046e7ae1-377f-4cd0-beba-28b76b5ceeac nodeName:}" failed. No retries permitted until 2026-07-30 16:37:18.562724002 +0000 UTC m=+66.165151610 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzpj5" (UniqueName: "kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5") pod "network-check-target-vd7hz" (UID: "046e7ae1-377f-4cd0-beba-28b76b5ceeac") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 16:36:46.994690 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:46.994654    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:46.994690 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:46.994692    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:46.994956 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:46.994654    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:46.994956 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.994778    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qzc6g" podUID="5643d33f-7080-4c6e-b5bf-bd7104cbae98"
Jul 30 16:36:46.994956 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.994843    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vd7hz" podUID="046e7ae1-377f-4cd0-beba-28b76b5ceeac"
Jul 30 16:36:46.994956 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:46.994937    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bwspl" podUID="7db173ed-c34e-43b4-9251-835908a35f70"
Jul 30 16:36:47.157187 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.157153    2569 generic.go:356] "Generic (PLEG): container finished" podID="8406d9dc-3cba-47f7-9448-4ebc5d75972f" containerID="2c5adcd2b89359fd53fd5a18a5087505dfe4cfd27c8e95beecd17f38c838e509" exitCode=0
Jul 30 16:36:47.157332 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.157199    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngd2j" event={"ID":"8406d9dc-3cba-47f7-9448-4ebc5d75972f","Type":"ContainerDied","Data":"2c5adcd2b89359fd53fd5a18a5087505dfe4cfd27c8e95beecd17f38c838e509"}
Jul 30 16:36:47.251716 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.251693    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-111.ec2.internal" event="NodeReady"
Jul 30 16:36:47.251826 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.251819    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 16:36:47.350030 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.350003    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kj7pk"]
Jul 30 16:36:47.362859 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.362830    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c68q5"]
Jul 30 16:36:47.363010 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.362967    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kj7pk"
Jul 30 16:36:47.366314 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.366294    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xkdnd\""
Jul 30 16:36:47.367024 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.367009    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 16:36:47.367093 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.367071    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 16:36:47.367145 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.367122    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 16:36:47.370866 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.370847    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.374253 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.374238    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 16:36:47.375104 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.375092    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 16:36:47.375669 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.375655    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-59qbk\""
Jul 30 16:36:47.388396 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.388363    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kj7pk"]
Jul 30 16:36:47.388944 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.388922    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c68q5"]
Jul 30 16:36:47.469103 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.469007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zfjw\" (UniqueName: \"kubernetes.io/projected/f4cb5628-b926-4f78-bfe1-c58a1850b600-kube-api-access-8zfjw\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.469103 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.469048    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmfn4\" (UniqueName: \"kubernetes.io/projected/7a2989d8-f863-4d75-8724-8a68cce35870-kube-api-access-mmfn4\") pod \"ingress-canary-kj7pk\" (UID: \"7a2989d8-f863-4d75-8724-8a68cce35870\") " pod="openshift-ingress-canary/ingress-canary-kj7pk"
Jul 30 16:36:47.469308 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.469122    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4cb5628-b926-4f78-bfe1-c58a1850b600-tmp-dir\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.469308 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.469202    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.469308 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.469259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2989d8-f863-4d75-8724-8a68cce35870-cert\") pod \"ingress-canary-kj7pk\" (UID: \"7a2989d8-f863-4d75-8724-8a68cce35870\") " pod="openshift-ingress-canary/ingress-canary-kj7pk"
Jul 30 16:36:47.469308 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.469286    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f4cb5628-b926-4f78-bfe1-c58a1850b600-config-volume\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.570373 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.570336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2989d8-f863-4d75-8724-8a68cce35870-cert\") pod \"ingress-canary-kj7pk\" (UID: \"7a2989d8-f863-4d75-8724-8a68cce35870\") " pod="openshift-ingress-canary/ingress-canary-kj7pk"
Jul 30 16:36:47.570373 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.570380    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f4cb5628-b926-4f78-bfe1-c58a1850b600-config-volume\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.571009 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.570425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zfjw\" (UniqueName: \"kubernetes.io/projected/f4cb5628-b926-4f78-bfe1-c58a1850b600-kube-api-access-8zfjw\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.571009 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.570446    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmfn4\" (UniqueName: \"kubernetes.io/projected/7a2989d8-f863-4d75-8724-8a68cce35870-kube-api-access-mmfn4\") pod \"ingress-canary-kj7pk\" (UID: \"7a2989d8-f863-4d75-8724-8a68cce35870\") " pod="openshift-ingress-canary/ingress-canary-kj7pk"
Jul 30 16:36:47.571009 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.570473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4cb5628-b926-4f78-bfe1-c58a1850b600-tmp-dir\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.571009 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.570496    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.571009 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:47.570587    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:47.571009 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:47.570674    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls podName:f4cb5628-b926-4f78-bfe1-c58a1850b600 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:48.070660276 +0000 UTC m=+35.673087885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls") pod "dns-default-c68q5" (UID: "f4cb5628-b926-4f78-bfe1-c58a1850b600") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:47.571009 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.570867    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4cb5628-b926-4f78-bfe1-c58a1850b600-tmp-dir\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.571270 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.571078    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f4cb5628-b926-4f78-bfe1-c58a1850b600-config-volume\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.574694 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.574669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2989d8-f863-4d75-8724-8a68cce35870-cert\") pod \"ingress-canary-kj7pk\" (UID: \"7a2989d8-f863-4d75-8724-8a68cce35870\") " pod="openshift-ingress-canary/ingress-canary-kj7pk"
Jul 30 16:36:47.583568 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.583545    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zfjw\" (UniqueName: \"kubernetes.io/projected/f4cb5628-b926-4f78-bfe1-c58a1850b600-kube-api-access-8zfjw\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:47.585252 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.585230    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmfn4\" (UniqueName: \"kubernetes.io/projected/7a2989d8-f863-4d75-8724-8a68cce35870-kube-api-access-mmfn4\") pod \"ingress-canary-kj7pk\" (UID: \"7a2989d8-f863-4d75-8724-8a68cce35870\") " pod="openshift-ingress-canary/ingress-canary-kj7pk"
Jul 30 16:36:47.700386 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.700348    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kj7pk"
Jul 30 16:36:47.891864 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:47.891835    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kj7pk"]
Jul 30 16:36:47.895787 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:47.895757    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2989d8_f863_4d75_8724_8a68cce35870.slice/crio-d1e5ce6868af2ff90e59e5c5844561dc16ef676269137304cd4d0e7a3bc1b774 WatchSource:0}: Error finding container d1e5ce6868af2ff90e59e5c5844561dc16ef676269137304cd4d0e7a3bc1b774: Status 404 returned error can't find the container with id d1e5ce6868af2ff90e59e5c5844561dc16ef676269137304cd4d0e7a3bc1b774
Jul 30 16:36:48.074030 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.073799    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:48.074030 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:48.073961    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:48.074208 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:48.074102    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls podName:f4cb5628-b926-4f78-bfe1-c58a1850b600 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:49.074086659 +0000 UTC m=+36.676514263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls") pod "dns-default-c68q5" (UID: "f4cb5628-b926-4f78-bfe1-c58a1850b600") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:48.162343 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.162308    2569 generic.go:356] "Generic (PLEG): container finished" podID="8406d9dc-3cba-47f7-9448-4ebc5d75972f" containerID="4af3f39651412158da0232c481328422ba53518de02a7ab871b3122e9db3c4ce" exitCode=0
Jul 30 16:36:48.162526 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.162380    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngd2j" event={"ID":"8406d9dc-3cba-47f7-9448-4ebc5d75972f","Type":"ContainerDied","Data":"4af3f39651412158da0232c481328422ba53518de02a7ab871b3122e9db3c4ce"}
Jul 30 16:36:48.163663 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.163637    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kj7pk" event={"ID":"7a2989d8-f863-4d75-8724-8a68cce35870","Type":"ContainerStarted","Data":"d1e5ce6868af2ff90e59e5c5844561dc16ef676269137304cd4d0e7a3bc1b774"}
Jul 30 16:36:48.994841 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.994804    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:36:48.995641 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.994803    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:36:48.995641 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.994809    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:36:48.998915 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.998894    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 16:36:48.999053 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.998930    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z2ksq\""
Jul 30 16:36:48.999112 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.999059    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 16:36:48.999112 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.999088    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2m4bh\""
Jul 30 16:36:48.999206 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.999167    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 16:36:48.999206 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:48.999179    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 16:36:49.083341 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:49.083300    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:49.083532 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:49.083465    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:49.083596 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:49.083543    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls podName:f4cb5628-b926-4f78-bfe1-c58a1850b600 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:51.083520835 +0000 UTC m=+38.685948443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls") pod "dns-default-c68q5" (UID: "f4cb5628-b926-4f78-bfe1-c58a1850b600") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:49.174098 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:49.174045    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ngd2j" event={"ID":"8406d9dc-3cba-47f7-9448-4ebc5d75972f","Type":"ContainerStarted","Data":"e44bc65a510f60fe1782e567f03791c84dde8a5e5c001d552cde9da18ada0891"}
Jul 30 16:36:49.197840 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:49.197647    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ngd2j" podStartSLOduration=7.505385941 podStartE2EDuration="36.197627736s" totalImagesPullingTime="28.692241795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:49.19593519 +0000 UTC m=+36.798362823" watchObservedRunningTime="2026-07-30 16:36:49.197627736 +0000 UTC m=+36.800055353"
Jul 30 16:36:51.100513 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:51.100428    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:51.100876 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:51.100586    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:51.100876 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:51.100664    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls podName:f4cb5628-b926-4f78-bfe1-c58a1850b600 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:55.100647536 +0000 UTC m=+42.703075141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls") pod "dns-default-c68q5" (UID: "f4cb5628-b926-4f78-bfe1-c58a1850b600") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:51.179711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:51.179677    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kj7pk" event={"ID":"7a2989d8-f863-4d75-8724-8a68cce35870","Type":"ContainerStarted","Data":"88b57789afaffb662f5ab73267f14189d3feccc9fc2ddf37a266fd7b5ea0b33c"}
Jul 30 16:36:51.200565 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:51.200512    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kj7pk" podStartSLOduration=1.9263303349999998 podStartE2EDuration="4.200496957s" totalImagesPullingTime="2.274166622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:51.199913946 +0000 UTC m=+38.802341574" watchObservedRunningTime="2026-07-30 16:36:51.200496957 +0000 UTC m=+38.802924583"
Jul 30 16:36:53.213052 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.212869    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59"]
Jul 30 16:36:53.246290 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.246262    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59"]
Jul 30 16:36:53.246413 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.246356    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59"
Jul 30 16:36:53.249230 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.249211    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-kbltz\""
Jul 30 16:36:53.413415 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.413387    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-khc6x"]
Jul 30 16:36:53.415913 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.415886    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb5nl\" (UniqueName: \"kubernetes.io/projected/542f5470-15aa-4b96-83c8-ad13264774d4-kube-api-access-nb5nl\") pod \"network-check-source-7d88cb55d8-vds59\" (UID: \"542f5470-15aa-4b96-83c8-ad13264774d4\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59"
Jul 30 16:36:53.428205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.428184    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.428493 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.428193    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-khc6x"]
Jul 30 16:36:53.430888 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.430865    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 30 16:36:53.430888 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.430887    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 16:36:53.431040 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.430865    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 30 16:36:53.431101 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.431085    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 16:36:53.431151 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.431130    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f6pk9\""
Jul 30 16:36:53.436468 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.436448    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 30 16:36:53.516281 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.516220    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"]
Jul 30 16:36:53.516402 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.516383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb5nl\" (UniqueName: \"kubernetes.io/projected/542f5470-15aa-4b96-83c8-ad13264774d4-kube-api-access-nb5nl\") pod \"network-check-source-7d88cb55d8-vds59\" (UID: \"542f5470-15aa-4b96-83c8-ad13264774d4\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59"
Jul 30 16:36:53.516463 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.516429    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-tmp\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.516463 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.516448    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-snapshots\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.516570 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.516484    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-serving-cert\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.516570 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.516548    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.516570 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.516565    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-service-ca-bundle\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.516733 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.516580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wb29\" (UniqueName: \"kubernetes.io/projected/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-kube-api-access-7wb29\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.537025 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.537000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb5nl\" (UniqueName: \"kubernetes.io/projected/542f5470-15aa-4b96-83c8-ad13264774d4-kube-api-access-nb5nl\") pod \"network-check-source-7d88cb55d8-vds59\" (UID: \"542f5470-15aa-4b96-83c8-ad13264774d4\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59"
Jul 30 16:36:53.537998 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.537982    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"]
Jul 30 16:36:53.538097 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.538085    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.540587 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.540568    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 16:36:53.540826 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.540812    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 16:36:53.541187 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.541170    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 16:36:53.541279 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.541187    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-m62gx\""
Jul 30 16:36:53.545640 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.545602    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 16:36:53.554935 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.554912    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59"
Jul 30 16:36:53.617377 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.617277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-tmp\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.617377 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.617324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-snapshots\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.617377 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.617351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-serving-cert\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.617664 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.617435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.617664 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.617462    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-service-ca-bundle\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.617664 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.617494    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wb29\" (UniqueName: \"kubernetes.io/projected/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-kube-api-access-7wb29\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.619098 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.618570    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-tmp\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.619839 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.619585    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-snapshots\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.619839 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.619791    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-service-ca-bundle\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.623084 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.620727    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-serving-cert\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.624477 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.624423    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.629780 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.629737    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wb29\" (UniqueName: \"kubernetes.io/projected/d11a3a2b-9176-4dc1-ad78-6b1002aa5947-kube-api-access-7wb29\") pod \"insights-operator-5885fdb759-khc6x\" (UID: \"d11a3a2b-9176-4dc1-ad78-6b1002aa5947\") " pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.690169 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.690139    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59"]
Jul 30 16:36:53.693930 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:53.693897    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod542f5470_15aa_4b96_83c8_ad13264774d4.slice/crio-6c781a6efef2f3fffce9b98a0ab5494f4efb0bd03e4fcc9049432f1db3906899 WatchSource:0}: Error finding container 6c781a6efef2f3fffce9b98a0ab5494f4efb0bd03e4fcc9049432f1db3906899: Status 404 returned error can't find the container with id 6c781a6efef2f3fffce9b98a0ab5494f4efb0bd03e4fcc9049432f1db3906899
Jul 30 16:36:53.718513 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.718487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-image-registry-private-configuration\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.718592 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.718518    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.718592 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.718535    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-trusted-ca\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.718711 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.718601    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-ca-trust-extracted\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.718764 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.718722    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-installation-pull-secrets\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.718816 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.718774    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-bound-sa-token\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.718869 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.718816    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wk2k\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-kube-api-access-7wk2k\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.718869 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.718861    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-certificates\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.737357 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.737334    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-khc6x"
Jul 30 16:36:53.819737 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.819706    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wk2k\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-kube-api-access-7wk2k\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.819737 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.819741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-certificates\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.819932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.819789    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-image-registry-private-configuration\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.819932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.819807    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.819932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.819823    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-trusted-ca\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.819932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.819859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-ca-trust-extracted\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.819932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.819898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-installation-pull-secrets\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.820160 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:53.819930    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 16:36:53.820160 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:53.819952    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84fdbcbbfc-mw8j5: secret "image-registry-tls" not found
Jul 30 16:36:53.820160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.819945    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-bound-sa-token\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.820160 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:53.820011    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls podName:093d93c7-e71b-44bf-bfcf-6b4a0fccc271 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:54.319990824 +0000 UTC m=+41.922418439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls") pod "image-registry-84fdbcbbfc-mw8j5" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271") : secret "image-registry-tls" not found
Jul 30 16:36:53.820625 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.820588    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-ca-trust-extracted\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.820910 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.820889    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-certificates\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.821462 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.821439    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-trusted-ca\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.822217 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.822190    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-image-registry-private-configuration\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.822489 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.822471    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-installation-pull-secrets\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.828880 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.828722    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-bound-sa-token\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.829254 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.829114    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wk2k\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-kube-api-access-7wk2k\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:53.874224 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:53.874189    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-khc6x"]
Jul 30 16:36:53.877125 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:53.877103    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11a3a2b_9176_4dc1_ad78_6b1002aa5947.slice/crio-1f4b26322bd2c5b03d43157bf7ab6684b2cca10920fc1c6e8963fd1ba11dcd9a WatchSource:0}: Error finding container 1f4b26322bd2c5b03d43157bf7ab6684b2cca10920fc1c6e8963fd1ba11dcd9a: Status 404 returned error can't find the container with id 1f4b26322bd2c5b03d43157bf7ab6684b2cca10920fc1c6e8963fd1ba11dcd9a
Jul 30 16:36:54.188134 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:54.188097    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59" event={"ID":"542f5470-15aa-4b96-83c8-ad13264774d4","Type":"ContainerStarted","Data":"6c781a6efef2f3fffce9b98a0ab5494f4efb0bd03e4fcc9049432f1db3906899"}
Jul 30 16:36:54.189361 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:54.189337    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-khc6x" event={"ID":"d11a3a2b-9176-4dc1-ad78-6b1002aa5947","Type":"ContainerStarted","Data":"1f4b26322bd2c5b03d43157bf7ab6684b2cca10920fc1c6e8963fd1ba11dcd9a"}
Jul 30 16:36:54.324451 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:54.324412    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:54.324845 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:54.324569    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 16:36:54.324845 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:54.324587    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84fdbcbbfc-mw8j5: secret "image-registry-tls" not found
Jul 30 16:36:54.324845 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:54.324658    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls podName:093d93c7-e71b-44bf-bfcf-6b4a0fccc271 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:55.324643328 +0000 UTC m=+42.927070935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls") pod "image-registry-84fdbcbbfc-mw8j5" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271") : secret "image-registry-tls" not found
Jul 30 16:36:55.130628 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:55.130579    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:36:55.130794 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:55.130730    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:36:55.130851 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:55.130797    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls podName:f4cb5628-b926-4f78-bfe1-c58a1850b600 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:03.130780104 +0000 UTC m=+50.733207713 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls") pod "dns-default-c68q5" (UID: "f4cb5628-b926-4f78-bfe1-c58a1850b600") : secret "dns-default-metrics-tls" not found
Jul 30 16:36:55.333347 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:55.333307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:55.333814 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:55.333468    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 16:36:55.333814 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:55.333493    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84fdbcbbfc-mw8j5: secret "image-registry-tls" not found
Jul 30 16:36:55.333814 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:55.333562    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls podName:093d93c7-e71b-44bf-bfcf-6b4a0fccc271 nodeName:}" failed. No retries permitted until 2026-07-30 16:36:57.333541535 +0000 UTC m=+44.935969158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls") pod "image-registry-84fdbcbbfc-mw8j5" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271") : secret "image-registry-tls" not found
Jul 30 16:36:57.349493 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:57.349461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:36:57.349849 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:57.349570    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 16:36:57.349849 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:57.349581    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84fdbcbbfc-mw8j5: secret "image-registry-tls" not found
Jul 30 16:36:57.349849 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:57.349664    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls podName:093d93c7-e71b-44bf-bfcf-6b4a0fccc271 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:01.349647774 +0000 UTC m=+48.952075379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls") pod "image-registry-84fdbcbbfc-mw8j5" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271") : secret "image-registry-tls" not found
Jul 30 16:36:58.201854 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.201805    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-khc6x" event={"ID":"d11a3a2b-9176-4dc1-ad78-6b1002aa5947","Type":"ContainerStarted","Data":"66d290298d967b7804b8ca0ce85b283300dfc3fd7b17519fc02a8b764423eeb2"}
Jul 30 16:36:58.203471 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.203443    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59" event={"ID":"542f5470-15aa-4b96-83c8-ad13264774d4","Type":"ContainerStarted","Data":"1deceab2737c0d9a481d49598cc64d0137e6ac24eb4a008ecc5000c64561d688"}
Jul 30 16:36:58.229617 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.229573    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-khc6x" podStartSLOduration=1.6144247059999999 podStartE2EDuration="5.229559882s" totalImagesPullingTime="3.615135176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:58.228559082 +0000 UTC m=+45.830986708" watchObservedRunningTime="2026-07-30 16:36:58.229559882 +0000 UTC m=+45.831987540"
Jul 30 16:36:58.267326 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.267268    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vds59" podStartSLOduration=1.473366491 podStartE2EDuration="5.267250174s" totalImagesPullingTime="3.793883683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:36:58.263048569 +0000 UTC m=+45.865476198" watchObservedRunningTime="2026-07-30 16:36:58.267250174 +0000 UTC m=+45.869677801"
Jul 30 16:36:58.353270 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.353242    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq"]
Jul 30 16:36:58.382250 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.382224    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq"]
Jul 30 16:36:58.382390 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.382311    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq"
Jul 30 16:36:58.385981 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.385957    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 30 16:36:58.386259 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.386242    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hs598\""
Jul 30 16:36:58.386259 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.386253    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 30 16:36:58.558237 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.558170    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnkwv\" (UniqueName: \"kubernetes.io/projected/213879ae-ddf4-40a6-bfe7-a24265d53411-kube-api-access-tnkwv\") pod \"migrator-6865c9967-v6prq\" (UID: \"213879ae-ddf4-40a6-bfe7-a24265d53411\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq"
Jul 30 16:36:58.659091 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.659060    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnkwv\" (UniqueName: \"kubernetes.io/projected/213879ae-ddf4-40a6-bfe7-a24265d53411-kube-api-access-tnkwv\") pod \"migrator-6865c9967-v6prq\" (UID: \"213879ae-ddf4-40a6-bfe7-a24265d53411\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq"
Jul 30 16:36:58.672793 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.672762    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnkwv\" (UniqueName: \"kubernetes.io/projected/213879ae-ddf4-40a6-bfe7-a24265d53411-kube-api-access-tnkwv\") pod \"migrator-6865c9967-v6prq\" (UID: \"213879ae-ddf4-40a6-bfe7-a24265d53411\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq"
Jul 30 16:36:58.691476 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.691446    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq"
Jul 30 16:36:58.868195 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.868118    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-74z6w"]
Jul 30 16:36:58.878101 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:36:58.878076    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213879ae_ddf4_40a6_bfe7_a24265d53411.slice/crio-d8da149da97644bc8cf79eff1427d29af2783402cc3cae041d5db06a8a61c03f WatchSource:0}: Error finding container d8da149da97644bc8cf79eff1427d29af2783402cc3cae041d5db06a8a61c03f: Status 404 returned error can't find the container with id d8da149da97644bc8cf79eff1427d29af2783402cc3cae041d5db06a8a61c03f
Jul 30 16:36:58.899092 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.899065    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq"]
Jul 30 16:36:58.899193 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.899105    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-74z6w"]
Jul 30 16:36:58.899193 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.899178    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:58.902160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.902143    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 16:36:58.902450 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.902433    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vspgg\""
Jul 30 16:36:58.902450 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:58.902444    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 16:36:59.062631 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.062585    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.062793 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.062656    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kff7\" (UniqueName: \"kubernetes.io/projected/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-api-access-9kff7\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.062793 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.062688    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.062793 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.062720    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a31939ac-a41d-44a3-b3ee-853c40acf93f-data-volume\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.062895 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.062818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a31939ac-a41d-44a3-b3ee-853c40acf93f-crio-socket\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.163904 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.163881    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a31939ac-a41d-44a3-b3ee-853c40acf93f-data-volume\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.164020 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.163949    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a31939ac-a41d-44a3-b3ee-853c40acf93f-crio-socket\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.164020 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.163973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.164020 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.164000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kff7\" (UniqueName: \"kubernetes.io/projected/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-api-access-9kff7\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.164158 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.164070    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.164158 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:59.164088    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 16:36:59.164262 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:59.164162    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-rbac-proxy-cm podName:a31939ac-a41d-44a3-b3ee-853c40acf93f nodeName:}" failed. No retries permitted until 2026-07-30 16:36:59.664141835 +0000 UTC m=+47.266569459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-74z6w" (UID: "a31939ac-a41d-44a3-b3ee-853c40acf93f") : configmap "kube-rbac-proxy" not found
Jul 30 16:36:59.164262 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.164172    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a31939ac-a41d-44a3-b3ee-853c40acf93f-crio-socket\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.164262 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.164187    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a31939ac-a41d-44a3-b3ee-853c40acf93f-data-volume\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.164262 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:59.164195    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:36:59.164262 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:59.164241    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls podName:a31939ac-a41d-44a3-b3ee-853c40acf93f nodeName:}" failed. No retries permitted until 2026-07-30 16:36:59.664228104 +0000 UTC m=+47.266655708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74z6w" (UID: "a31939ac-a41d-44a3-b3ee-853c40acf93f") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:36:59.173040 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.173022    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kff7\" (UniqueName: \"kubernetes.io/projected/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-api-access-9kff7\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.206421 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.206391    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq" event={"ID":"213879ae-ddf4-40a6-bfe7-a24265d53411","Type":"ContainerStarted","Data":"d8da149da97644bc8cf79eff1427d29af2783402cc3cae041d5db06a8a61c03f"}
Jul 30 16:36:59.668400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.668365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.668763 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:36:59.668417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:36:59.668763 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:59.668492    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 16:36:59.668763 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:59.668554    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:36:59.668763 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:59.668563    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-rbac-proxy-cm podName:a31939ac-a41d-44a3-b3ee-853c40acf93f nodeName:}" failed. No retries permitted until 2026-07-30 16:37:00.668542193 +0000 UTC m=+48.270969817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-74z6w" (UID: "a31939ac-a41d-44a3-b3ee-853c40acf93f") : configmap "kube-rbac-proxy" not found
Jul 30 16:36:59.668763 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:36:59.668588    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls podName:a31939ac-a41d-44a3-b3ee-853c40acf93f nodeName:}" failed. No retries permitted until 2026-07-30 16:37:00.668579384 +0000 UTC m=+48.271007001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74z6w" (UID: "a31939ac-a41d-44a3-b3ee-853c40acf93f") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:00.676739 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:00.676713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:37:00.677018 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:00.676784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:37:00.677018 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:00.676925    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:00.677018 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:00.676981    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls podName:a31939ac-a41d-44a3-b3ee-853c40acf93f nodeName:}" failed. No retries permitted until 2026-07-30 16:37:02.676964574 +0000 UTC m=+50.279392185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74z6w" (UID: "a31939ac-a41d-44a3-b3ee-853c40acf93f") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:00.677137 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:00.677120    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a31939ac-a41d-44a3-b3ee-853c40acf93f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:37:01.212761 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:01.212730    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq" event={"ID":"213879ae-ddf4-40a6-bfe7-a24265d53411","Type":"ContainerStarted","Data":"a12a47ea5650dd57c990ef871521906574e73c87e9a9f5275e4c93ec81584f5f"}
Jul 30 16:37:01.212921 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:01.212778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq" event={"ID":"213879ae-ddf4-40a6-bfe7-a24265d53411","Type":"ContainerStarted","Data":"110010c2a1111e0ae8075c5442e71d6454ff7b14805c498024c993fa10ed281b"}
Jul 30 16:37:01.233003 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:01.232960    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v6prq" podStartSLOduration=1.445584953 podStartE2EDuration="3.232945649s" totalImagesPullingTime="1.787360696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:01.232765336 +0000 UTC m=+48.835192962" watchObservedRunningTime="2026-07-30 16:37:01.232945649 +0000 UTC m=+48.835373274"
Jul 30 16:37:01.382205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:01.382182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:37:01.382351 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:01.382332    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 16:37:01.382388 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:01.382353    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84fdbcbbfc-mw8j5: secret "image-registry-tls" not found
Jul 30 16:37:01.382421 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:01.382401    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls podName:093d93c7-e71b-44bf-bfcf-6b4a0fccc271 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:09.382386901 +0000 UTC m=+56.984814505 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls") pod "image-registry-84fdbcbbfc-mw8j5" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271") : secret "image-registry-tls" not found
Jul 30 16:37:01.445620 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:01.445581    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-79c74_264e7227-dd74-4d87-9f65-107983fb8538/dns-node-resolver/0.log"
Jul 30 16:37:02.446127 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:02.446096    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kw2nz_0c926555-bc21-4e17-98de-15f8d25b951a/node-ca/0.log"
Jul 30 16:37:02.693206 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:02.693178    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:37:02.693348 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:02.693312    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:02.693395 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:02.693385    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls podName:a31939ac-a41d-44a3-b3ee-853c40acf93f nodeName:}" failed. No retries permitted until 2026-07-30 16:37:06.69336965 +0000 UTC m=+54.295797254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74z6w" (UID: "a31939ac-a41d-44a3-b3ee-853c40acf93f") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:03.197850 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:03.197813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:37:03.198011 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:03.197955    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 16:37:03.198064 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:03.198015    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls podName:f4cb5628-b926-4f78-bfe1-c58a1850b600 nodeName:}" failed. No retries permitted until 2026-07-30 16:37:19.198000015 +0000 UTC m=+66.800427619 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls") pod "dns-default-c68q5" (UID: "f4cb5628-b926-4f78-bfe1-c58a1850b600") : secret "dns-default-metrics-tls" not found
Jul 30 16:37:03.244588 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:03.244565    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kj7pk_7a2989d8-f863-4d75-8724-8a68cce35870/serve-healthcheck-canary/0.log"
Jul 30 16:37:03.852548 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:03.852518    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-v6prq_213879ae-ddf4-40a6-bfe7-a24265d53411/migrator/0.log"
Jul 30 16:37:04.045207 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:04.045175    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-v6prq_213879ae-ddf4-40a6-bfe7-a24265d53411/graceful-termination/0.log"
Jul 30 16:37:06.724899 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:06.724867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:37:06.725490 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:06.725461    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:06.725574 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:06.725563    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls podName:a31939ac-a41d-44a3-b3ee-853c40acf93f nodeName:}" failed. No retries permitted until 2026-07-30 16:37:14.725540985 +0000 UTC m=+62.327968596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-74z6w" (UID: "a31939ac-a41d-44a3-b3ee-853c40acf93f") : secret "insights-runtime-extractor-tls" not found
Jul 30 16:37:09.447264 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:09.447233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:37:09.449443 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:09.449424    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") pod \"image-registry-84fdbcbbfc-mw8j5\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") " pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:37:09.748931 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:09.748858    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:37:09.879893 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:09.879866    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"]
Jul 30 16:37:09.882513 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:09.882487    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093d93c7_e71b_44bf_bfcf_6b4a0fccc271.slice/crio-5d97bc7aaac8bb9c01d3dfb63ba48d770ae15b89acb7cb56c7bb3a2fa468e87c WatchSource:0}: Error finding container 5d97bc7aaac8bb9c01d3dfb63ba48d770ae15b89acb7cb56c7bb3a2fa468e87c: Status 404 returned error can't find the container with id 5d97bc7aaac8bb9c01d3dfb63ba48d770ae15b89acb7cb56c7bb3a2fa468e87c
Jul 30 16:37:10.237347 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:10.237313    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5" event={"ID":"093d93c7-e71b-44bf-bfcf-6b4a0fccc271","Type":"ContainerStarted","Data":"68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6"}
Jul 30 16:37:10.237514 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:10.237372    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5" event={"ID":"093d93c7-e71b-44bf-bfcf-6b4a0fccc271","Type":"ContainerStarted","Data":"5d97bc7aaac8bb9c01d3dfb63ba48d770ae15b89acb7cb56c7bb3a2fa468e87c"}
Jul 30 16:37:10.237514 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:10.237442    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:37:10.261666 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:10.261600    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5" podStartSLOduration=17.261586756 podStartE2EDuration="17.261586756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:10.260967662 +0000 UTC m=+57.863395289" watchObservedRunningTime="2026-07-30 16:37:10.261586756 +0000 UTC m=+57.864014360"
Jul 30 16:37:13.156277 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:13.156249    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-44szc"
Jul 30 16:37:14.793804 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:14.793769    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:37:14.795923 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:14.795905    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a31939ac-a41d-44a3-b3ee-853c40acf93f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-74z6w\" (UID: \"a31939ac-a41d-44a3-b3ee-853c40acf93f\") " pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:37:14.809775 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:14.809754    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vspgg\""
Jul 30 16:37:14.817846 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:14.817832    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-74z6w"
Jul 30 16:37:14.936749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:14.936722    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-74z6w"]
Jul 30 16:37:14.939557 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:14.939528    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda31939ac_a41d_44a3_b3ee_853c40acf93f.slice/crio-9a1decf9e74c6010400af69711cdf5c8df52ac2c258dbaa1580b61f9d0b2ab1e WatchSource:0}: Error finding container 9a1decf9e74c6010400af69711cdf5c8df52ac2c258dbaa1580b61f9d0b2ab1e: Status 404 returned error can't find the container with id 9a1decf9e74c6010400af69711cdf5c8df52ac2c258dbaa1580b61f9d0b2ab1e
Jul 30 16:37:15.253791 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:15.253759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74z6w" event={"ID":"a31939ac-a41d-44a3-b3ee-853c40acf93f","Type":"ContainerStarted","Data":"9a1decf9e74c6010400af69711cdf5c8df52ac2c258dbaa1580b61f9d0b2ab1e"}
Jul 30 16:37:18.626553 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.626507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:37:18.627026 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.626594    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:37:18.627026 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.626649    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:37:18.629664 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.629594    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzpj5\" (UniqueName: \"kubernetes.io/projected/046e7ae1-377f-4cd0-beba-28b76b5ceeac-kube-api-access-jzpj5\") pod \"network-check-target-vd7hz\" (UID: \"046e7ae1-377f-4cd0-beba-28b76b5ceeac\") " pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:37:18.633203 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.633173    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 16:37:18.633341 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.633319    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 16:37:18.639966 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.639936    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5643d33f-7080-4c6e-b5bf-bd7104cbae98-metrics-certs\") pod \"network-metrics-daemon-qzc6g\" (UID: \"5643d33f-7080-4c6e-b5bf-bd7104cbae98\") " pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:37:18.640083 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.640044    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7db173ed-c34e-43b4-9251-835908a35f70-original-pull-secret\") pod \"global-pull-secret-syncer-bwspl\" (UID: \"7db173ed-c34e-43b4-9251-835908a35f70\") " pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:37:18.710515 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.710484    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2m4bh\""
Jul 30 16:37:18.716710 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.716681    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z2ksq\""
Jul 30 16:37:18.718779 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.718754    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:37:18.721734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.721707    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bwspl"
Jul 30 16:37:18.724833 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:18.724809    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qzc6g"
Jul 30 16:37:19.230883 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:19.230753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:37:19.232941 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:19.232917    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4cb5628-b926-4f78-bfe1-c58a1850b600-metrics-tls\") pod \"dns-default-c68q5\" (UID: \"f4cb5628-b926-4f78-bfe1-c58a1850b600\") " pod="openshift-dns/dns-default-c68q5"
Jul 30 16:37:19.507824 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:19.507757    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-59qbk\""
Jul 30 16:37:19.515292 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:19.515265    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c68q5"
Jul 30 16:37:20.018784 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.018747    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qzc6g"]
Jul 30 16:37:20.022557 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:20.022508    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5643d33f_7080_4c6e_b5bf_bd7104cbae98.slice/crio-469e375a5c93b9a01581a098acc53824ae7460e060dcffcde79d20f45a41c656 WatchSource:0}: Error finding container 469e375a5c93b9a01581a098acc53824ae7460e060dcffcde79d20f45a41c656: Status 404 returned error can't find the container with id 469e375a5c93b9a01581a098acc53824ae7460e060dcffcde79d20f45a41c656
Jul 30 16:37:20.037843 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.037809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bwspl"]
Jul 30 16:37:20.041338 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:20.041297    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db173ed_c34e_43b4_9251_835908a35f70.slice/crio-bfe176306ab33bc7b5fbd2cc5dd3f823e3a5d8051007294e14141753870baac3 WatchSource:0}: Error finding container bfe176306ab33bc7b5fbd2cc5dd3f823e3a5d8051007294e14141753870baac3: Status 404 returned error can't find the container with id bfe176306ab33bc7b5fbd2cc5dd3f823e3a5d8051007294e14141753870baac3
Jul 30 16:37:20.055655 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.055554    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c68q5"]
Jul 30 16:37:20.058000 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:20.057973    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cb5628_b926_4f78_bfe1_c58a1850b600.slice/crio-24793260d0bf9dec5873e8aa7649fe8a949ba6c973564279ef436fbbd22ad818 WatchSource:0}: Error finding container 24793260d0bf9dec5873e8aa7649fe8a949ba6c973564279ef436fbbd22ad818: Status 404 returned error can't find the container with id 24793260d0bf9dec5873e8aa7649fe8a949ba6c973564279ef436fbbd22ad818
Jul 30 16:37:20.067877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.067839    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vd7hz"]
Jul 30 16:37:20.071121 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:20.071096    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod046e7ae1_377f_4cd0_beba_28b76b5ceeac.slice/crio-7161982b522ff227726d25d853d75f5e0eff7f585bd948bbf65bbafcb20d0ded WatchSource:0}: Error finding container 7161982b522ff227726d25d853d75f5e0eff7f585bd948bbf65bbafcb20d0ded: Status 404 returned error can't find the container with id 7161982b522ff227726d25d853d75f5e0eff7f585bd948bbf65bbafcb20d0ded
Jul 30 16:37:20.272252 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.272222    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vd7hz" event={"ID":"046e7ae1-377f-4cd0-beba-28b76b5ceeac","Type":"ContainerStarted","Data":"f9a6c07fb2ab614f2eb3dbf0bede5bd14ddacc04303d3232909c82111d3fbb45"}
Jul 30 16:37:20.272452 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.272269    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vd7hz" event={"ID":"046e7ae1-377f-4cd0-beba-28b76b5ceeac","Type":"ContainerStarted","Data":"7161982b522ff227726d25d853d75f5e0eff7f585bd948bbf65bbafcb20d0ded"}
Jul 30 16:37:20.272452 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.272333    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:37:20.273435 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.273403    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qzc6g" event={"ID":"5643d33f-7080-4c6e-b5bf-bd7104cbae98","Type":"ContainerStarted","Data":"469e375a5c93b9a01581a098acc53824ae7460e060dcffcde79d20f45a41c656"}
Jul 30 16:37:20.274795 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.274775    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74z6w" event={"ID":"a31939ac-a41d-44a3-b3ee-853c40acf93f","Type":"ContainerStarted","Data":"c08082592106966fecbe12ad77e5340580ad3b153f58f0960dd4841bc89b5269"}
Jul 30 16:37:20.275784 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.275768    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c68q5" event={"ID":"f4cb5628-b926-4f78-bfe1-c58a1850b600","Type":"ContainerStarted","Data":"24793260d0bf9dec5873e8aa7649fe8a949ba6c973564279ef436fbbd22ad818"}
Jul 30 16:37:20.278319 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.278297    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bwspl" event={"ID":"7db173ed-c34e-43b4-9251-835908a35f70","Type":"ContainerStarted","Data":"bfe176306ab33bc7b5fbd2cc5dd3f823e3a5d8051007294e14141753870baac3"}
Jul 30 16:37:20.290881 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:20.290820    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vd7hz" podStartSLOduration=67.290800387 podStartE2EDuration="1m7.290800387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:20.290297553 +0000 UTC m=+67.892725181" watchObservedRunningTime="2026-07-30 16:37:20.290800387 +0000 UTC m=+67.893227992"
Jul 30 16:37:22.289328 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.289288    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qzc6g" event={"ID":"5643d33f-7080-4c6e-b5bf-bd7104cbae98","Type":"ContainerStarted","Data":"56af229f41e77af4842ffb74450fbffb7dfcc01a8d78d3a61927918578c7b54e"}
Jul 30 16:37:22.554400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.554293    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"]
Jul 30 16:37:22.561410 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.561381    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"
Jul 30 16:37:22.566263 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.566236    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fjbq2\""
Jul 30 16:37:22.566401 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.566284    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 16:37:22.567213 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.567125    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 16:37:22.567487 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.567319    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 16:37:22.567487 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.567440    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 30 16:37:22.571722 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.571680    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"]
Jul 30 16:37:22.631031 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.630997    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-ltd5p"]
Jul 30 16:37:22.633542 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.633489    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-ltd5p"
Jul 30 16:37:22.638389 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.638362    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 16:37:22.638725 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.638640    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 16:37:22.638835 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.638784    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-swn49\""
Jul 30 16:37:22.658964 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.658926    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/816a5276-29e8-4614-a823-7f4118254c7a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7db76d4c96-pv7nv\" (UID: \"816a5276-29e8-4614-a823-7f4118254c7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"
Jul 30 16:37:22.659192 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.659020    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blhz4\" (UniqueName: \"kubernetes.io/projected/816a5276-29e8-4614-a823-7f4118254c7a-kube-api-access-blhz4\") pod \"managed-serviceaccount-addon-agent-7db76d4c96-pv7nv\" (UID: \"816a5276-29e8-4614-a823-7f4118254c7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"
Jul 30 16:37:22.665412 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.664128    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-ltd5p"]
Jul 30 16:37:22.665412 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.665234    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"]
Jul 30 16:37:22.724785 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.724747    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54b44df6c6-6dkrn"]
Jul 30 16:37:22.727406 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.727373    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.752036 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.752002    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54b44df6c6-6dkrn"]
Jul 30 16:37:22.759424 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.759386    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bpn8\" (UniqueName: \"kubernetes.io/projected/5e9801d4-cabf-4d45-9526-f002e1451590-kube-api-access-7bpn8\") pod \"downloads-7c97dc4555-ltd5p\" (UID: \"5e9801d4-cabf-4d45-9526-f002e1451590\") " pod="openshift-console/downloads-7c97dc4555-ltd5p"
Jul 30 16:37:22.759596 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.759455    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/816a5276-29e8-4614-a823-7f4118254c7a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7db76d4c96-pv7nv\" (UID: \"816a5276-29e8-4614-a823-7f4118254c7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"
Jul 30 16:37:22.759596 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.759493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blhz4\" (UniqueName: \"kubernetes.io/projected/816a5276-29e8-4614-a823-7f4118254c7a-kube-api-access-blhz4\") pod \"managed-serviceaccount-addon-agent-7db76d4c96-pv7nv\" (UID: \"816a5276-29e8-4614-a823-7f4118254c7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"
Jul 30 16:37:22.763007 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.762980    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/816a5276-29e8-4614-a823-7f4118254c7a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7db76d4c96-pv7nv\" (UID: \"816a5276-29e8-4614-a823-7f4118254c7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"
Jul 30 16:37:22.776565 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.776529    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blhz4\" (UniqueName: \"kubernetes.io/projected/816a5276-29e8-4614-a823-7f4118254c7a-kube-api-access-blhz4\") pod \"managed-serviceaccount-addon-agent-7db76d4c96-pv7nv\" (UID: \"816a5276-29e8-4614-a823-7f4118254c7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"
Jul 30 16:37:22.860072 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.859999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94g7q\" (UniqueName: \"kubernetes.io/projected/e1d983da-cbef-4822-8d92-f9b9accdfb4b-kube-api-access-94g7q\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.860072 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.860040    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1d983da-cbef-4822-8d92-f9b9accdfb4b-bound-sa-token\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.860285 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.860080    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e1d983da-cbef-4822-8d92-f9b9accdfb4b-registry-certificates\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.860285 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.860117    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e1d983da-cbef-4822-8d92-f9b9accdfb4b-image-registry-private-configuration\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.860285 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.860180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e1d983da-cbef-4822-8d92-f9b9accdfb4b-registry-tls\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.860285 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.860217    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bpn8\" (UniqueName: \"kubernetes.io/projected/5e9801d4-cabf-4d45-9526-f002e1451590-kube-api-access-7bpn8\") pod \"downloads-7c97dc4555-ltd5p\" (UID: \"5e9801d4-cabf-4d45-9526-f002e1451590\") " pod="openshift-console/downloads-7c97dc4555-ltd5p"
Jul 30 16:37:22.860285 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.860237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e1d983da-cbef-4822-8d92-f9b9accdfb4b-ca-trust-extracted\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.860285 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.860257    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1d983da-cbef-4822-8d92-f9b9accdfb4b-trusted-ca\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.860285 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.860280    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e1d983da-cbef-4822-8d92-f9b9accdfb4b-installation-pull-secrets\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.870696 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.870596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bpn8\" (UniqueName: \"kubernetes.io/projected/5e9801d4-cabf-4d45-9526-f002e1451590-kube-api-access-7bpn8\") pod \"downloads-7c97dc4555-ltd5p\" (UID: \"5e9801d4-cabf-4d45-9526-f002e1451590\") " pod="openshift-console/downloads-7c97dc4555-ltd5p"
Jul 30 16:37:22.882314 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.882280    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"
Jul 30 16:37:22.945574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.945404    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-ltd5p"
Jul 30 16:37:22.960701 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.960668    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94g7q\" (UniqueName: \"kubernetes.io/projected/e1d983da-cbef-4822-8d92-f9b9accdfb4b-kube-api-access-94g7q\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.960701 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.960714    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1d983da-cbef-4822-8d92-f9b9accdfb4b-bound-sa-token\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.960933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.960750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e1d983da-cbef-4822-8d92-f9b9accdfb4b-registry-certificates\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.960933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.960791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e1d983da-cbef-4822-8d92-f9b9accdfb4b-image-registry-private-configuration\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.960933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.960835    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e1d983da-cbef-4822-8d92-f9b9accdfb4b-registry-tls\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.960933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.960869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e1d983da-cbef-4822-8d92-f9b9accdfb4b-ca-trust-extracted\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.960933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.960900    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1d983da-cbef-4822-8d92-f9b9accdfb4b-trusted-ca\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.960933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.960926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e1d983da-cbef-4822-8d92-f9b9accdfb4b-installation-pull-secrets\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.961826 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.961483    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e1d983da-cbef-4822-8d92-f9b9accdfb4b-ca-trust-extracted\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.962354 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.962321    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1d983da-cbef-4822-8d92-f9b9accdfb4b-trusted-ca\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.962670 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.962620    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e1d983da-cbef-4822-8d92-f9b9accdfb4b-registry-certificates\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.964617 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.964557    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e1d983da-cbef-4822-8d92-f9b9accdfb4b-installation-pull-secrets\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.965122 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.965102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e1d983da-cbef-4822-8d92-f9b9accdfb4b-registry-tls\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.966052 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.966027    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e1d983da-cbef-4822-8d92-f9b9accdfb4b-image-registry-private-configuration\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.984808 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.984745    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1d983da-cbef-4822-8d92-f9b9accdfb4b-bound-sa-token\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:22.987656 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:22.987628    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94g7q\" (UniqueName: \"kubernetes.io/projected/e1d983da-cbef-4822-8d92-f9b9accdfb4b-kube-api-access-94g7q\") pod \"image-registry-54b44df6c6-6dkrn\" (UID: \"e1d983da-cbef-4822-8d92-f9b9accdfb4b\") " pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:23.039787 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:23.039752    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:25.821274 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:25.821212    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv"]
Jul 30 16:37:25.823489 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:25.823445    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod816a5276_29e8_4614_a823_7f4118254c7a.slice/crio-c880f8f70ef189b014243c21d2c45f7c6cbbff9eac2ba466ec89843c1ebb38d7 WatchSource:0}: Error finding container c880f8f70ef189b014243c21d2c45f7c6cbbff9eac2ba466ec89843c1ebb38d7: Status 404 returned error can't find the container with id c880f8f70ef189b014243c21d2c45f7c6cbbff9eac2ba466ec89843c1ebb38d7
Jul 30 16:37:25.846036 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:25.846004    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-ltd5p"]
Jul 30 16:37:25.849634 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:25.849558    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54b44df6c6-6dkrn"]
Jul 30 16:37:25.853389 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:25.853349    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e9801d4_cabf_4d45_9526_f002e1451590.slice/crio-7719a5644d98500a4d77c44778ae3b738e54fcff11956095793ca0876ebd83ed WatchSource:0}: Error finding container 7719a5644d98500a4d77c44778ae3b738e54fcff11956095793ca0876ebd83ed: Status 404 returned error can't find the container with id 7719a5644d98500a4d77c44778ae3b738e54fcff11956095793ca0876ebd83ed
Jul 30 16:37:25.854728 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:25.854250    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d983da_cbef_4822_8d92_f9b9accdfb4b.slice/crio-904c7c2ff31e6e02528c858e0da786e7aea9020637ecd900a96281339deece50 WatchSource:0}: Error finding container 904c7c2ff31e6e02528c858e0da786e7aea9020637ecd900a96281339deece50: Status 404 returned error can't find the container with id 904c7c2ff31e6e02528c858e0da786e7aea9020637ecd900a96281339deece50
Jul 30 16:37:26.306524 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.306480    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bwspl" event={"ID":"7db173ed-c34e-43b4-9251-835908a35f70","Type":"ContainerStarted","Data":"f363db09cfb82b3ee1494a15381930727e576fdc7067e394962dbb97e47bf529"}
Jul 30 16:37:26.308089 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.308057    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-ltd5p" event={"ID":"5e9801d4-cabf-4d45-9526-f002e1451590","Type":"ContainerStarted","Data":"7719a5644d98500a4d77c44778ae3b738e54fcff11956095793ca0876ebd83ed"}
Jul 30 16:37:26.309397 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.309373    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv" event={"ID":"816a5276-29e8-4614-a823-7f4118254c7a","Type":"ContainerStarted","Data":"c880f8f70ef189b014243c21d2c45f7c6cbbff9eac2ba466ec89843c1ebb38d7"}
Jul 30 16:37:26.311066 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.311037    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn" event={"ID":"e1d983da-cbef-4822-8d92-f9b9accdfb4b","Type":"ContainerStarted","Data":"097e50a2bd70db2259eeda519692aae5cfeaddd3bf97aab12e59c68da8c3b470"}
Jul 30 16:37:26.311188 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.311080    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn" event={"ID":"e1d983da-cbef-4822-8d92-f9b9accdfb4b","Type":"ContainerStarted","Data":"904c7c2ff31e6e02528c858e0da786e7aea9020637ecd900a96281339deece50"}
Jul 30 16:37:26.311188 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.311123    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:26.313056 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.313032    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qzc6g" event={"ID":"5643d33f-7080-4c6e-b5bf-bd7104cbae98","Type":"ContainerStarted","Data":"dc7c42219cf2894e1efe20cc2846f56ce12c0b9495a7463b05979d027ee1f049"}
Jul 30 16:37:26.314797 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.314776    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74z6w" event={"ID":"a31939ac-a41d-44a3-b3ee-853c40acf93f","Type":"ContainerStarted","Data":"2158cd8bea27cba904dfb9b6c57e4472a54c9565546b1b78c40d4319da722366"}
Jul 30 16:37:26.316623 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.316585    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c68q5" event={"ID":"f4cb5628-b926-4f78-bfe1-c58a1850b600","Type":"ContainerStarted","Data":"231bb123b8436be379eba46f7e06c0074b7b3cf69a047516fda15cf9f5526c20"}
Jul 30 16:37:26.316727 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.316652    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c68q5" event={"ID":"f4cb5628-b926-4f78-bfe1-c58a1850b600","Type":"ContainerStarted","Data":"c85a15f126bc171472e2b83d60eef34631dd238b02298fb3e71c3f15a3a21117"}
Jul 30 16:37:26.316788 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.316731    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c68q5"
Jul 30 16:37:26.325199 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.325115    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bwspl" podStartSLOduration=67.729520312 podStartE2EDuration="1m13.325097395s" totalImagesPullingTime="5.595577083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:26.323682874 +0000 UTC m=+73.926110501" watchObservedRunningTime="2026-07-30 16:37:26.325097395 +0000 UTC m=+73.927525020"
Jul 30 16:37:26.340873 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.340810    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qzc6g" podStartSLOduration=71.750667746 podStartE2EDuration="1m13.340792915s" totalImagesPullingTime="1.590125169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:26.339806965 +0000 UTC m=+73.942234591" watchObservedRunningTime="2026-07-30 16:37:26.340792915 +0000 UTC m=+73.943220540"
Jul 30 16:37:26.359137 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.359069    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c68q5" podStartSLOduration=33.785169493 podStartE2EDuration="39.359047181s" totalImagesPullingTime="5.573877688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:26.358315897 +0000 UTC m=+73.960743514" watchObservedRunningTime="2026-07-30 16:37:26.359047181 +0000 UTC m=+73.961474809"
Jul 30 16:37:26.378866 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:26.378808    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn" podStartSLOduration=4.3787855669999995 podStartE2EDuration="4.378785567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:26.378710471 +0000 UTC m=+73.981138119" watchObservedRunningTime="2026-07-30 16:37:26.378785567 +0000 UTC m=+73.981213195"
Jul 30 16:37:32.345787 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:32.345746    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-74z6w" event={"ID":"a31939ac-a41d-44a3-b3ee-853c40acf93f","Type":"ContainerStarted","Data":"6cf3961fb537b26b2f52aed7ba11f7b033f0bf0a1dcf8864012a9255b07ba58e"}
Jul 30 16:37:32.347397 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:32.347356    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv" event={"ID":"816a5276-29e8-4614-a823-7f4118254c7a","Type":"ContainerStarted","Data":"9fb51853b6ff7cadc37f5029feb6017ab3a2021ad0009906f66794f8c1322738"}
Jul 30 16:37:32.372105 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:32.372033    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-74z6w" podStartSLOduration=17.874963035 podStartE2EDuration="34.372017845s" totalImagesPullingTime="16.49705481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 16:36:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:32.37200906 +0000 UTC m=+79.974436685" watchObservedRunningTime="2026-07-30 16:37:32.372017845 +0000 UTC m=+79.974445473"
Jul 30 16:37:32.671728 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:32.671680    2569 patch_prober.go:28] interesting pod/image-registry-84fdbcbbfc-mw8j5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 30 16:37:32.671905 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:32.671739    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5" podUID="093d93c7-e71b-44bf-bfcf-6b4a0fccc271" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 16:37:36.329047 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:36.328941    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c68q5"
Jul 30 16:37:36.348301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:36.348253    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db76d4c96-pv7nv" podStartSLOduration=8.508306001 podStartE2EDuration="14.348239208s" totalImagesPullingTime="5.839933207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:32.39407024 +0000 UTC m=+79.996497866" watchObservedRunningTime="2026-07-30 16:37:36.348239208 +0000 UTC m=+83.950666891"
Jul 30 16:37:37.151637 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.151581    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b4d787965-hrh7p"]
Jul 30 16:37:37.154150 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.154127    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.156808 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.156780    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 16:37:37.157025 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.156790    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 16:37:37.157753 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.157700    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 16:37:37.157997 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.157978    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 16:37:37.158084 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.158066    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4sdqp\""
Jul 30 16:37:37.158188 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.158174    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 16:37:37.161909 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.161888    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 16:37:37.166092 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.166063    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b4d787965-hrh7p"]
Jul 30 16:37:37.280473 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.280437    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr7sl\" (UniqueName: \"kubernetes.io/projected/67243d84-c09b-40d4-a6c1-aa2422713b76-kube-api-access-jr7sl\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.280473 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.280482    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-serving-cert\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.280738 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.280534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-oauth-serving-cert\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.280738 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.280581    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-oauth-config\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.280738 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.280630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-trusted-ca-bundle\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.280738 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.280650    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-service-ca\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.280738 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.280723    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-console-config\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.381443 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.381410    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-serving-cert\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.381925 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.381481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-oauth-serving-cert\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.381925 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.381506    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-oauth-config\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.381925 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.381529    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-trusted-ca-bundle\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.381925 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.381550    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-service-ca\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.381925 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.381587    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-console-config\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.381925 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.381625    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr7sl\" (UniqueName: \"kubernetes.io/projected/67243d84-c09b-40d4-a6c1-aa2422713b76-kube-api-access-jr7sl\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.382342 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.382263    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-oauth-serving-cert\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.382480 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.382459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-service-ca\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.382641 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.382599    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-console-config\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.382731 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.382649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-trusted-ca-bundle\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.384156 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.384135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-serving-cert\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.384296 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.384280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-oauth-config\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.391281 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.391261    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr7sl\" (UniqueName: \"kubernetes.io/projected/67243d84-c09b-40d4-a6c1-aa2422713b76-kube-api-access-jr7sl\") pod \"console-7b4d787965-hrh7p\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") " pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.463565 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.463475    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:37.597934 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:37.597906    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b4d787965-hrh7p"]
Jul 30 16:37:39.050237 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.050198    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-645xr"]
Jul 30 16:37:39.053341 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.053314    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.057293 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.057208    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 30 16:37:39.057293 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.057233    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 30 16:37:39.057293 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.057244    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 16:37:39.057293 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.057257    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-v7gdl\""
Jul 30 16:37:39.057592 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.057310    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 16:37:39.057592 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.057244    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 16:37:39.065016 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.064996    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-645xr"]
Jul 30 16:37:39.196911 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.196873    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.197104 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.196959    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.197104 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.196980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.197104 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.197023    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfnnq\" (UniqueName: \"kubernetes.io/projected/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-kube-api-access-xfnnq\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.297586 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.297548    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfnnq\" (UniqueName: \"kubernetes.io/projected/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-kube-api-access-xfnnq\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.297762 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.297596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.297762 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.297667    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.297762 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.297690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.297935 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:39.297807    2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 30 16:37:39.297935 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:39.297871    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-prometheus-operator-tls podName:52f328e8-7b11-4103-bcd1-203f6b4bbdfe nodeName:}" failed. No retries permitted until 2026-07-30 16:37:39.797848984 +0000 UTC m=+87.400276596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-645xr" (UID: "52f328e8-7b11-4103-bcd1-203f6b4bbdfe") : secret "prometheus-operator-tls" not found
Jul 30 16:37:39.298368 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.298346    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.300418 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.300365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.306905 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.306884    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfnnq\" (UniqueName: \"kubernetes.io/projected/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-kube-api-access-xfnnq\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.802004 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.801964    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.804843 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.804810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/52f328e8-7b11-4103-bcd1-203f6b4bbdfe-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-645xr\" (UID: \"52f328e8-7b11-4103-bcd1-203f6b4bbdfe\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:39.966035 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:39.966000    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr"
Jul 30 16:37:42.670389 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:42.670360    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:37:44.364896 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:44.364859    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67243d84_c09b_40d4_a6c1_aa2422713b76.slice/crio-a90cb610f7182d5714a988c327e994253eda7278963894cae9d55d6cd484bc04 WatchSource:0}: Error finding container a90cb610f7182d5714a988c327e994253eda7278963894cae9d55d6cd484bc04: Status 404 returned error can't find the container with id a90cb610f7182d5714a988c327e994253eda7278963894cae9d55d6cd484bc04
Jul 30 16:37:44.392106 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:44.392065    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4d787965-hrh7p" event={"ID":"67243d84-c09b-40d4-a6c1-aa2422713b76","Type":"ContainerStarted","Data":"a90cb610f7182d5714a988c327e994253eda7278963894cae9d55d6cd484bc04"}
Jul 30 16:37:44.488256 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:44.488225    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-645xr"]
Jul 30 16:37:44.491265 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:44.491238    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f328e8_7b11_4103_bcd1_203f6b4bbdfe.slice/crio-499a1c7c3bf2586828dcc71c64506f32b2ca790720d0be729a8ba6a47e3ccbc3 WatchSource:0}: Error finding container 499a1c7c3bf2586828dcc71c64506f32b2ca790720d0be729a8ba6a47e3ccbc3: Status 404 returned error can't find the container with id 499a1c7c3bf2586828dcc71c64506f32b2ca790720d0be729a8ba6a47e3ccbc3
Jul 30 16:37:45.399378 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:45.399283    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr" event={"ID":"52f328e8-7b11-4103-bcd1-203f6b4bbdfe","Type":"ContainerStarted","Data":"499a1c7c3bf2586828dcc71c64506f32b2ca790720d0be729a8ba6a47e3ccbc3"}
Jul 30 16:37:45.402221 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:45.402171    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-ltd5p" event={"ID":"5e9801d4-cabf-4d45-9526-f002e1451590","Type":"ContainerStarted","Data":"d487d0bebe817380dcf62be907b1ea41ab1979369685f68b00dbb475a9870df6"}
Jul 30 16:37:45.403876 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:45.403563    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-ltd5p"
Jul 30 16:37:45.423708 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:45.423587    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-ltd5p" podStartSLOduration=4.480471437 podStartE2EDuration="23.423566662s" totalImagesPullingTime="18.943095225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:45.422976115 +0000 UTC m=+93.025403741" watchObservedRunningTime="2026-07-30 16:37:45.423566662 +0000 UTC m=+93.025994289"
Jul 30 16:37:45.425301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:45.425243    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-ltd5p"
Jul 30 16:37:47.331020 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:47.330965    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54b44df6c6-6dkrn"
Jul 30 16:37:47.689247 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:47.689188    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5" podUID="093d93c7-e71b-44bf-bfcf-6b4a0fccc271" containerName="registry" containerID="cri-o://68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6" gracePeriod=30
Jul 30 16:37:48.414354 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.414328    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:37:48.418326 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.418292    2569 generic.go:356] "Generic (PLEG): container finished" podID="093d93c7-e71b-44bf-bfcf-6b4a0fccc271" containerID="68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6" exitCode=0
Jul 30 16:37:48.418451 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.418342    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"
Jul 30 16:37:48.418451 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.418386    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5" event={"ID":"093d93c7-e71b-44bf-bfcf-6b4a0fccc271","Type":"ContainerDied","Data":"68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6"}
Jul 30 16:37:48.418562 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.418474    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84fdbcbbfc-mw8j5" event={"ID":"093d93c7-e71b-44bf-bfcf-6b4a0fccc271","Type":"ContainerDied","Data":"5d97bc7aaac8bb9c01d3dfb63ba48d770ae15b89acb7cb56c7bb3a2fa468e87c"}
Jul 30 16:37:48.418562 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.418490    2569 scope.go:117] "RemoveContainer" containerID="68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6"
Jul 30 16:37:48.420254 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.420219    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr" event={"ID":"52f328e8-7b11-4103-bcd1-203f6b4bbdfe","Type":"ContainerStarted","Data":"caacbf0fc6af37609322689108cddd87247794823f821d73174caff75a3d0c96"}
Jul 30 16:37:48.481301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481010    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-image-registry-private-configuration\") pod \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") "
Jul 30 16:37:48.481301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481078    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-certificates\") pod \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") "
Jul 30 16:37:48.481301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481115    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") pod \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") "
Jul 30 16:37:48.481301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481146    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-bound-sa-token\") pod \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") "
Jul 30 16:37:48.481301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481177    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-ca-trust-extracted\") pod \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") "
Jul 30 16:37:48.481301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481228    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-installation-pull-secrets\") pod \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") "
Jul 30 16:37:48.481301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481255    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wk2k\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-kube-api-access-7wk2k\") pod \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") "
Jul 30 16:37:48.481301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481279    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-trusted-ca\") pod \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\" (UID: \"093d93c7-e71b-44bf-bfcf-6b4a0fccc271\") "
Jul 30 16:37:48.481878 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481425    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "093d93c7-e71b-44bf-bfcf-6b4a0fccc271" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:37:48.481878 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.481831    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "093d93c7-e71b-44bf-bfcf-6b4a0fccc271" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:37:48.489315 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.488938    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-kube-api-access-7wk2k" (OuterVolumeSpecName: "kube-api-access-7wk2k") pod "093d93c7-e71b-44bf-bfcf-6b4a0fccc271" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271"). InnerVolumeSpecName "kube-api-access-7wk2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:37:48.489315 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.489079    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "093d93c7-e71b-44bf-bfcf-6b4a0fccc271" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:37:48.489315 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.489213    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "093d93c7-e71b-44bf-bfcf-6b4a0fccc271" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:37:48.489932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.489812    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "093d93c7-e71b-44bf-bfcf-6b4a0fccc271" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:37:48.489932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.489881    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "093d93c7-e71b-44bf-bfcf-6b4a0fccc271" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:37:48.493206 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.493174    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "093d93c7-e71b-44bf-bfcf-6b4a0fccc271" (UID: "093d93c7-e71b-44bf-bfcf-6b4a0fccc271"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:37:48.582801 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.582764    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-trusted-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:37:48.582801 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.582796    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-image-registry-private-configuration\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:37:48.582801 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.582806    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-certificates\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:37:48.583034 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.582819    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-registry-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:37:48.583034 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.582832    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-bound-sa-token\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:37:48.583034 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.582840    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-ca-trust-extracted\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:37:48.583034 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.582847    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-installation-pull-secrets\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:37:48.583034 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.582859    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wk2k\" (UniqueName: \"kubernetes.io/projected/093d93c7-e71b-44bf-bfcf-6b4a0fccc271-kube-api-access-7wk2k\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:37:48.592586 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.592553    2569 scope.go:117] "RemoveContainer" containerID="68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6"
Jul 30 16:37:48.593045 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:37:48.593012    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6\": container with ID starting with 68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6 not found: ID does not exist" containerID="68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6"
Jul 30 16:37:48.593142 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.593057    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6"} err="failed to get container status \"68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6\": rpc error: code = NotFound desc = could not find container \"68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6\": container with ID starting with 68a08b63e808ec69af7f012a3b94c3baa1e9b0642d9a31d23dbff8be1b89b5a6 not found: ID does not exist"
Jul 30 16:37:48.741147 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.741084    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"]
Jul 30 16:37:48.746723 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:48.746699    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84fdbcbbfc-mw8j5"]
Jul 30 16:37:49.000932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:49.000855    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="093d93c7-e71b-44bf-bfcf-6b4a0fccc271" path="/var/lib/kubelet/pods/093d93c7-e71b-44bf-bfcf-6b4a0fccc271/volumes"
Jul 30 16:37:49.426933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:49.426890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr" event={"ID":"52f328e8-7b11-4103-bcd1-203f6b4bbdfe","Type":"ContainerStarted","Data":"f0072eec0b8b90663e591c6903771aa3830295291a99d9975a5819e9a0072680"}
Jul 30 16:37:49.429668 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:49.429637    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4d787965-hrh7p" event={"ID":"67243d84-c09b-40d4-a6c1-aa2422713b76","Type":"ContainerStarted","Data":"3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e"}
Jul 30 16:37:49.446866 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:49.446812    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-645xr" podStartSLOduration=6.707579595 podStartE2EDuration="10.446789847s" totalImagesPullingTime="3.739210252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:49.445209343 +0000 UTC m=+97.047636970" watchObservedRunningTime="2026-07-30 16:37:49.446789847 +0000 UTC m=+97.049217452"
Jul 30 16:37:49.463459 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:49.463404    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b4d787965-hrh7p" podStartSLOduration=8.229051144 podStartE2EDuration="12.463386137s" totalImagesPullingTime="4.234334993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:49.462176918 +0000 UTC m=+97.064604546" watchObservedRunningTime="2026-07-30 16:37:49.463386137 +0000 UTC m=+97.065813763"
Jul 30 16:37:51.287032 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.286996    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vd7hz"
Jul 30 16:37:51.487850 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.487816    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lxvvr"]
Jul 30 16:37:51.488155 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.488128    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="093d93c7-e71b-44bf-bfcf-6b4a0fccc271" containerName="registry"
Jul 30 16:37:51.488155 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.488150    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="093d93c7-e71b-44bf-bfcf-6b4a0fccc271" containerName="registry"
Jul 30 16:37:51.488324 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.488208    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="093d93c7-e71b-44bf-bfcf-6b4a0fccc271" containerName="registry"
Jul 30 16:37:51.547250 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.547172    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.550044 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.550017    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 16:37:51.550179 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.550089    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 16:37:51.550248 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.550225    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 16:37:51.550315 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.550297    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rzhcm\""
Jul 30 16:37:51.608982 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.608941    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-textfile\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.609169 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.608993    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/648cf43a-f478-40e8-92af-40e987c5de73-metrics-client-ca\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.609169 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.609052    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/648cf43a-f478-40e8-92af-40e987c5de73-sys\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.609169 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.609076    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-tls\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.609169 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.609145    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.609369 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.609203    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-wtmp\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.609369 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.609224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.609369 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.609254    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/648cf43a-f478-40e8-92af-40e987c5de73-root\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.609369 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.609312    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkbp7\" (UniqueName: \"kubernetes.io/projected/648cf43a-f478-40e8-92af-40e987c5de73-kube-api-access-xkbp7\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.710300 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710263    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkbp7\" (UniqueName: \"kubernetes.io/projected/648cf43a-f478-40e8-92af-40e987c5de73-kube-api-access-xkbp7\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.710484 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-textfile\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.710633 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710585    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/648cf43a-f478-40e8-92af-40e987c5de73-metrics-client-ca\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.710716 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710676    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/648cf43a-f478-40e8-92af-40e987c5de73-sys\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.710716 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710704    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-tls\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.710821 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710737    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.710821 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710767    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-textfile\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.710821 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710773    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/648cf43a-f478-40e8-92af-40e987c5de73-sys\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.710821 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-wtmp\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.711014 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710840    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.711014 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/648cf43a-f478-40e8-92af-40e987c5de73-root\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.711014 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710907    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-wtmp\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.711014 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.710948    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/648cf43a-f478-40e8-92af-40e987c5de73-root\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.711279 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.711256    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/648cf43a-f478-40e8-92af-40e987c5de73-metrics-client-ca\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.711682 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.711664    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.713770 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.713745    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.713856 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.713796    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/648cf43a-f478-40e8-92af-40e987c5de73-node-exporter-tls\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.721266 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.721243    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkbp7\" (UniqueName: \"kubernetes.io/projected/648cf43a-f478-40e8-92af-40e987c5de73-kube-api-access-xkbp7\") pod \"node-exporter-lxvvr\" (UID: \"648cf43a-f478-40e8-92af-40e987c5de73\") " pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.858128 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:51.858050    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxvvr"
Jul 30 16:37:51.903451 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:51.903421    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648cf43a_f478_40e8_92af_40e987c5de73.slice/crio-6a8a57a5762ae1f4b320119a9fa0011f4c8d41bd1da24dcf535b352b1275aafe WatchSource:0}: Error finding container 6a8a57a5762ae1f4b320119a9fa0011f4c8d41bd1da24dcf535b352b1275aafe: Status 404 returned error can't find the container with id 6a8a57a5762ae1f4b320119a9fa0011f4c8d41bd1da24dcf535b352b1275aafe
Jul 30 16:37:52.443389 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.443355    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxvvr" event={"ID":"648cf43a-f478-40e8-92af-40e987c5de73","Type":"ContainerStarted","Data":"6a8a57a5762ae1f4b320119a9fa0011f4c8d41bd1da24dcf535b352b1275aafe"}
Jul 30 16:37:52.547756 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.547721    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 16:37:52.571525 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.571485    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.572570 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.572539    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 16:37:52.574459 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.574435    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 30 16:37:52.574912 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.574728    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 30 16:37:52.575093 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.575061    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 30 16:37:52.575287 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.575268    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 30 16:37:52.575588 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.575222    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 30 16:37:52.575720 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.575703    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 30 16:37:52.575921 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.575650    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 30 16:37:52.576082 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.575995    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 30 16:37:52.576255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.576242    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 30 16:37:52.576330 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.576318    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8d95g\""
Jul 30 16:37:52.618550 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.618507    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.618819 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.618785    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.618911 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.618845    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.618911 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.618873    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.618911 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.618903    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.619064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.618930    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.619064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.618960    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.619064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.618988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-config-out\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.619064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.619011    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.619064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.619040    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-web-config\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.619283 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.619098    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-766sg\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-kube-api-access-766sg\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.619283 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.619151    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.619283 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.619175    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720550 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720464    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-web-config\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720550 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720536    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-766sg\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-kube-api-access-766sg\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720761 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720587    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720761 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720761 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720672    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720761 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720697    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720761 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720975 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720975 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720975 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720803    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720975 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720828    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720975 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-config-out\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.720975 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.720862    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.722284 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.722003    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.723354 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.722906    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.723354 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.722913    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.724529 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.724507    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-config-volume\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.725982 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.725882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.727143 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.727122    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-web-config\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.727513 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.727457    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.727799 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.727772    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.727895 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.727878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-config-out\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.730580 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.730530    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.731079 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.731058    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.731169 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.731061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.731718 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.731696    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-766sg\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-kube-api-access-766sg\") pod \"alertmanager-main-0\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:52.889404 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:52.889371    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:37:53.350571 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:53.350537    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 16:37:53.354089 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:53.354058    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e19e9a0_b291_46dd_baaa_b347e892801f.slice/crio-4590e15c223e8920548d08f904b8de3e48ab6154e5d48815a06e9dc920b65569 WatchSource:0}: Error finding container 4590e15c223e8920548d08f904b8de3e48ab6154e5d48815a06e9dc920b65569: Status 404 returned error can't find the container with id 4590e15c223e8920548d08f904b8de3e48ab6154e5d48815a06e9dc920b65569
Jul 30 16:37:53.448876 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:53.448793    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerStarted","Data":"4590e15c223e8920548d08f904b8de3e48ab6154e5d48815a06e9dc920b65569"}
Jul 30 16:37:54.454189 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:54.454148    2569 generic.go:356] "Generic (PLEG): container finished" podID="648cf43a-f478-40e8-92af-40e987c5de73" containerID="009099b29d90a41b670aa6208f8061d4c156fdd21b58bff2735297b08ef8eb62" exitCode=0
Jul 30 16:37:54.454643 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:54.454236    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxvvr" event={"ID":"648cf43a-f478-40e8-92af-40e987c5de73","Type":"ContainerDied","Data":"009099b29d90a41b670aa6208f8061d4c156fdd21b58bff2735297b08ef8eb62"}
Jul 30 16:37:55.459796 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.459764    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxvvr" event={"ID":"648cf43a-f478-40e8-92af-40e987c5de73","Type":"ContainerStarted","Data":"37ea2907b559425d77e63b5dc2032bf27315fc0336dd2ff864d48e1a3f1befa6"}
Jul 30 16:37:55.460309 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.459818    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxvvr" event={"ID":"648cf43a-f478-40e8-92af-40e987c5de73","Type":"ContainerStarted","Data":"7874083386f8b230159fe68c281540759c50ec70bdcd8d4e7a4f846e3d929a8c"}
Jul 30 16:37:55.461339 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.461312    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerID="9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c" exitCode=0
Jul 30 16:37:55.461418 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.461397    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerDied","Data":"9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c"}
Jul 30 16:37:55.480684 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.480580    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lxvvr" podStartSLOduration=2.789042901 podStartE2EDuration="4.480564437s" totalImagesPullingTime="1.588312571s" totalInitContainerRuntime="103.208965ms" isStatefulPod=true podCreationTimestamp="2026-07-30 16:37:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:55.479481465 +0000 UTC m=+103.081909090" watchObservedRunningTime="2026-07-30 16:37:55.480564437 +0000 UTC m=+103.082992062"
Jul 30 16:37:55.808209 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.808132    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-65495d8796-2pnkx"]
Jul 30 16:37:55.843704 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.843667    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65495d8796-2pnkx"]
Jul 30 16:37:55.843882 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.843785    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:55.846721 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.846692    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 30 16:37:55.846890 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.846699    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-57z77\""
Jul 30 16:37:55.846890 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.846702    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 30 16:37:55.846890 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.846809    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bqsa7a6kpu71e\""
Jul 30 16:37:55.847661 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.847644    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 30 16:37:55.848135 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.848116    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 30 16:37:55.949040 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.948998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rljs\" (UniqueName: \"kubernetes.io/projected/9cdb2448-daf4-41c7-88c0-3677852f671b-kube-api-access-5rljs\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:55.949208 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.949046    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9cdb2448-daf4-41c7-88c0-3677852f671b-secret-metrics-server-client-certs\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:55.949208 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.949159    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9cdb2448-daf4-41c7-88c0-3677852f671b-secret-metrics-server-tls\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:55.949208 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.949198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cdb2448-daf4-41c7-88c0-3677852f671b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:55.949305 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.949224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9cdb2448-daf4-41c7-88c0-3677852f671b-audit-log\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:55.949305 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.949266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9cdb2448-daf4-41c7-88c0-3677852f671b-metrics-server-audit-profiles\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:55.949305 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:55.949299    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9cdb2448-daf4-41c7-88c0-3677852f671b-client-ca-bundle\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.050158 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.050119    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9cdb2448-daf4-41c7-88c0-3677852f671b-metrics-server-audit-profiles\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.050158 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.050166    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9cdb2448-daf4-41c7-88c0-3677852f671b-client-ca-bundle\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.050361 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.050286    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rljs\" (UniqueName: \"kubernetes.io/projected/9cdb2448-daf4-41c7-88c0-3677852f671b-kube-api-access-5rljs\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.050361 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.050330    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9cdb2448-daf4-41c7-88c0-3677852f671b-secret-metrics-server-client-certs\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.050361 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.050357    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9cdb2448-daf4-41c7-88c0-3677852f671b-secret-metrics-server-tls\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.050466 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.050398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cdb2448-daf4-41c7-88c0-3677852f671b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.050661 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.050593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9cdb2448-daf4-41c7-88c0-3677852f671b-audit-log\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.051077 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.051053    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cdb2448-daf4-41c7-88c0-3677852f671b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.051175 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.051158    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9cdb2448-daf4-41c7-88c0-3677852f671b-audit-log\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.051236 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.051177    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9cdb2448-daf4-41c7-88c0-3677852f671b-metrics-server-audit-profiles\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.052722 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.052688    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9cdb2448-daf4-41c7-88c0-3677852f671b-secret-metrics-server-tls\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.052874 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.052855    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9cdb2448-daf4-41c7-88c0-3677852f671b-secret-metrics-server-client-certs\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.052932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.052859    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9cdb2448-daf4-41c7-88c0-3677852f671b-client-ca-bundle\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.058731 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.058672    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rljs\" (UniqueName: \"kubernetes.io/projected/9cdb2448-daf4-41c7-88c0-3677852f671b-kube-api-access-5rljs\") pod \"metrics-server-65495d8796-2pnkx\" (UID: \"9cdb2448-daf4-41c7-88c0-3677852f671b\") " pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.153285 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.153252    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:37:56.202895 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.202864    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b4d787965-hrh7p"]
Jul 30 16:37:56.335056 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.335024    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65495d8796-2pnkx"]
Jul 30 16:37:56.338240 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:56.338212    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cdb2448_daf4_41c7_88c0_3677852f671b.slice/crio-d445df2e2f646ddc15e3d600464f794c26905c8f86e17f54e21509e2b8d0ccda WatchSource:0}: Error finding container d445df2e2f646ddc15e3d600464f794c26905c8f86e17f54e21509e2b8d0ccda: Status 404 returned error can't find the container with id d445df2e2f646ddc15e3d600464f794c26905c8f86e17f54e21509e2b8d0ccda
Jul 30 16:37:56.468191 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:56.468152    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65495d8796-2pnkx" event={"ID":"9cdb2448-daf4-41c7-88c0-3677852f671b","Type":"ContainerStarted","Data":"d445df2e2f646ddc15e3d600464f794c26905c8f86e17f54e21509e2b8d0ccda"}
Jul 30 16:37:57.463809 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.463773    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:37:57.721455 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.721358    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 16:37:57.751386 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.751304    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 16:37:57.751574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.751425    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.756879 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.756849    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 30 16:37:57.757581 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.757393    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 30 16:37:57.757581 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.757473    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 30 16:37:57.757581 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.757506    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 30 16:37:57.757994 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.757974    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 30 16:37:57.758097 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.758025    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bxw24\""
Jul 30 16:37:57.758198 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.758181    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 30 16:37:57.759083 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.759061    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 30 16:37:57.760823 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.759279    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 30 16:37:57.760823 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.759483    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 30 16:37:57.760823 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.759735    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 30 16:37:57.760823 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.759944    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4oggtuj18a2q0\""
Jul 30 16:37:57.760823 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.760141    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 30 16:37:57.762317 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.762112    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 30 16:37:57.763980 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.763815    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 30 16:37:57.766657 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766632    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.766770 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766694    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.766770 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.766770 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766753    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.766941 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.766941 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766803    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.766941 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766871    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.766941 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766923    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767140 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766948    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767140 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.766984    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-config\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767140 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767140 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767034    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767140 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767062    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767140 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj4xs\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-kube-api-access-dj4xs\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767140 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767124    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767471 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767159    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767471 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767187    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767471 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767242    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767471 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767471 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.767471 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.767419    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 30 16:37:57.870380 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870345    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.870561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870392    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.870561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.870561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.870561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870464    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-config\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.870561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870482    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.870561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870500    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.870561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870526    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.870561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj4xs\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-kube-api-access-dj4xs\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.870904 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.870575    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.871316 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.871278    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.871717 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.871598    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.871717 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.871654    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.871878 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.871733    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.871979    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.872018    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.872064    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.872116    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.872169    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.872207    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.872232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.872258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.872283    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.872344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.872343    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.873448 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.873103    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.873448 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.873183    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.874266 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.873926    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.874547 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.874491    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.876178 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.876149    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.877487 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.876811    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.877487 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.877040    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.877487 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.877224    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.877487 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.877226    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-config\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.877487 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.877443    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.878730 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.878705    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.879082 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.879061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-config-out\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.879152 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.879114    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.880201 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.880180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.881214 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.881158    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-web-config\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:57.881690 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:57.881647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj4xs\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-kube-api-access-dj4xs\") pod \"prometheus-k8s-0\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:58.065496 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:58.065415    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:37:58.290235 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:58.289591    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 16:37:58.293358 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:37:58.293330    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ea597c_c679_4fb9_810a_44953e63eb14.slice/crio-a904a3ec1e546e8e769faf27ebdd3a24fff0daa01f2e6b2812bd8dbb9445584f WatchSource:0}: Error finding container a904a3ec1e546e8e769faf27ebdd3a24fff0daa01f2e6b2812bd8dbb9445584f: Status 404 returned error can't find the container with id a904a3ec1e546e8e769faf27ebdd3a24fff0daa01f2e6b2812bd8dbb9445584f
Jul 30 16:37:58.481379 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:58.481345    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerStarted","Data":"11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8"}
Jul 30 16:37:58.481506 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:58.481422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerStarted","Data":"a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10"}
Jul 30 16:37:58.482552 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:58.482517    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerStarted","Data":"a904a3ec1e546e8e769faf27ebdd3a24fff0daa01f2e6b2812bd8dbb9445584f"}
Jul 30 16:37:59.491358 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:59.491325    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65495d8796-2pnkx" event={"ID":"9cdb2448-daf4-41c7-88c0-3677852f671b","Type":"ContainerStarted","Data":"56974644392d7abf2c715750a9330c76d2b559eec0ae83d4a6f0390714b8b1cc"}
Jul 30 16:37:59.493252 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:59.493213    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerID="66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340" exitCode=0
Jul 30 16:37:59.493385 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:59.493307    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerDied","Data":"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"}
Jul 30 16:37:59.496935 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:59.496850    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerStarted","Data":"7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6"}
Jul 30 16:37:59.496935 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:59.496906    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerStarted","Data":"e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9"}
Jul 30 16:37:59.496935 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:59.496935    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerStarted","Data":"cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4"}
Jul 30 16:37:59.514108 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:37:59.514053    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-65495d8796-2pnkx" podStartSLOduration=2.378151426 podStartE2EDuration="4.514031425s" totalImagesPullingTime="2.135879999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:37:59.513509398 +0000 UTC m=+107.115937026" watchObservedRunningTime="2026-07-30 16:37:59.514031425 +0000 UTC m=+107.116459053"
Jul 30 16:38:00.505574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:00.505486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerStarted","Data":"ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955"}
Jul 30 16:38:00.543527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:00.543459    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.714674619 podStartE2EDuration="8.543442402s" totalImagesPullingTime="5.816209699s" totalInitContainerRuntime="12.558084ms" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:38:00.539939704 +0000 UTC m=+108.142367331" watchObservedRunningTime="2026-07-30 16:38:00.543442402 +0000 UTC m=+108.145870032"
Jul 30 16:38:03.520242 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:03.520214    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerStarted","Data":"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"}
Jul 30 16:38:04.532243 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:04.532210    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerStarted","Data":"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"}
Jul 30 16:38:06.542337 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:06.542297    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerStarted","Data":"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"}
Jul 30 16:38:06.542805 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:06.542380    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerStarted","Data":"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"}
Jul 30 16:38:06.542805 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:06.542418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerStarted","Data":"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"}
Jul 30 16:38:06.542805 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:06.542453    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerStarted","Data":"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"}
Jul 30 16:38:06.577998 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:06.577945    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.446057421 podStartE2EDuration="9.577925245s" totalImagesPullingTime="6.120912195s" totalInitContainerRuntime="10.955629ms" isStatefulPod=false podCreationTimestamp="2026-07-30 16:37:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:38:06.576104545 +0000 UTC m=+114.178532170" watchObservedRunningTime="2026-07-30 16:38:06.577925245 +0000 UTC m=+114.180352871"
Jul 30 16:38:08.065943 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:08.065903    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:38:16.153746 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:16.153710    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:38:16.154128 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:16.153766    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:38:18.589646 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:18.589552    2569 generic.go:356] "Generic (PLEG): container finished" podID="d11a3a2b-9176-4dc1-ad78-6b1002aa5947" containerID="66d290298d967b7804b8ca0ce85b283300dfc3fd7b17519fc02a8b764423eeb2" exitCode=0
Jul 30 16:38:18.590162 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:18.589641    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-khc6x" event={"ID":"d11a3a2b-9176-4dc1-ad78-6b1002aa5947","Type":"ContainerDied","Data":"66d290298d967b7804b8ca0ce85b283300dfc3fd7b17519fc02a8b764423eeb2"}
Jul 30 16:38:18.590162 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:18.590079    2569 scope.go:117] "RemoveContainer" containerID="66d290298d967b7804b8ca0ce85b283300dfc3fd7b17519fc02a8b764423eeb2"
Jul 30 16:38:19.595913 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:19.595879    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-khc6x" event={"ID":"d11a3a2b-9176-4dc1-ad78-6b1002aa5947","Type":"ContainerStarted","Data":"6cdc2c719888c9ce8441669fea63398030c4e3c80245e8079793aacb21e71bc6"}
Jul 30 16:38:21.221627 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.221570    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b4d787965-hrh7p" podUID="67243d84-c09b-40d4-a6c1-aa2422713b76" containerName="console" containerID="cri-o://3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e" gracePeriod=15
Jul 30 16:38:21.488409 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.488355    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:38:21.589056 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589026    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-oauth-serving-cert\") pod \"67243d84-c09b-40d4-a6c1-aa2422713b76\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") "
Jul 30 16:38:21.589205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589079    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr7sl\" (UniqueName: \"kubernetes.io/projected/67243d84-c09b-40d4-a6c1-aa2422713b76-kube-api-access-jr7sl\") pod \"67243d84-c09b-40d4-a6c1-aa2422713b76\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") "
Jul 30 16:38:21.589205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589105    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-trusted-ca-bundle\") pod \"67243d84-c09b-40d4-a6c1-aa2422713b76\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") "
Jul 30 16:38:21.589205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589122    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-service-ca\") pod \"67243d84-c09b-40d4-a6c1-aa2422713b76\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") "
Jul 30 16:38:21.589205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589144    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-serving-cert\") pod \"67243d84-c09b-40d4-a6c1-aa2422713b76\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") "
Jul 30 16:38:21.589205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589157    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-console-config\") pod \"67243d84-c09b-40d4-a6c1-aa2422713b76\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") "
Jul 30 16:38:21.589205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589185    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-oauth-config\") pod \"67243d84-c09b-40d4-a6c1-aa2422713b76\" (UID: \"67243d84-c09b-40d4-a6c1-aa2422713b76\") "
Jul 30 16:38:21.589519 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589492    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "67243d84-c09b-40d4-a6c1-aa2422713b76" (UID: "67243d84-c09b-40d4-a6c1-aa2422713b76"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:38:21.589589 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589559    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "67243d84-c09b-40d4-a6c1-aa2422713b76" (UID: "67243d84-c09b-40d4-a6c1-aa2422713b76"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:38:21.589808 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589781    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-service-ca" (OuterVolumeSpecName: "service-ca") pod "67243d84-c09b-40d4-a6c1-aa2422713b76" (UID: "67243d84-c09b-40d4-a6c1-aa2422713b76"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:38:21.589900 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.589833    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-console-config" (OuterVolumeSpecName: "console-config") pod "67243d84-c09b-40d4-a6c1-aa2422713b76" (UID: "67243d84-c09b-40d4-a6c1-aa2422713b76"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:38:21.591324 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.591296    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67243d84-c09b-40d4-a6c1-aa2422713b76-kube-api-access-jr7sl" (OuterVolumeSpecName: "kube-api-access-jr7sl") pod "67243d84-c09b-40d4-a6c1-aa2422713b76" (UID: "67243d84-c09b-40d4-a6c1-aa2422713b76"). InnerVolumeSpecName "kube-api-access-jr7sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:38:21.591324 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.591319    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "67243d84-c09b-40d4-a6c1-aa2422713b76" (UID: "67243d84-c09b-40d4-a6c1-aa2422713b76"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:38:21.591433 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.591390    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "67243d84-c09b-40d4-a6c1-aa2422713b76" (UID: "67243d84-c09b-40d4-a6c1-aa2422713b76"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:38:21.604773 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.604751    2569 generic.go:356] "Generic (PLEG): container finished" podID="67243d84-c09b-40d4-a6c1-aa2422713b76" containerID="3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e" exitCode=0
Jul 30 16:38:21.604866 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.604794    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4d787965-hrh7p" event={"ID":"67243d84-c09b-40d4-a6c1-aa2422713b76","Type":"ContainerDied","Data":"3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e"}
Jul 30 16:38:21.604866 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.604801    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4d787965-hrh7p"
Jul 30 16:38:21.604866 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.604831    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4d787965-hrh7p" event={"ID":"67243d84-c09b-40d4-a6c1-aa2422713b76","Type":"ContainerDied","Data":"a90cb610f7182d5714a988c327e994253eda7278963894cae9d55d6cd484bc04"}
Jul 30 16:38:21.604866 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.604851    2569 scope.go:117] "RemoveContainer" containerID="3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e"
Jul 30 16:38:21.613024 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.613010    2569 scope.go:117] "RemoveContainer" containerID="3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e"
Jul 30 16:38:21.613265 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:38:21.613246    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e\": container with ID starting with 3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e not found: ID does not exist" containerID="3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e"
Jul 30 16:38:21.613330 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.613274    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e"} err="failed to get container status \"3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e\": rpc error: code = NotFound desc = could not find container \"3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e\": container with ID starting with 3a83627ce51174e55f44521fc14219406e320e9623a4f05cfd1fc05eedfa377e not found: ID does not exist"
Jul 30 16:38:21.630939 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.630917    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b4d787965-hrh7p"]
Jul 30 16:38:21.643553 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.643522    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b4d787965-hrh7p"]
Jul 30 16:38:21.690828 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.690806    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-oauth-serving-cert\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:38:21.690828 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.690825    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr7sl\" (UniqueName: \"kubernetes.io/projected/67243d84-c09b-40d4-a6c1-aa2422713b76-kube-api-access-jr7sl\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:38:21.690932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.690836    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-trusted-ca-bundle\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:38:21.690932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.690845    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:38:21.690932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.690854    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-serving-cert\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:38:21.690932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.690862    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67243d84-c09b-40d4-a6c1-aa2422713b76-console-config\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:38:21.690932 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:21.690870    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67243d84-c09b-40d4-a6c1-aa2422713b76-console-oauth-config\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:38:22.999549 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:22.999488    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67243d84-c09b-40d4-a6c1-aa2422713b76" path="/var/lib/kubelet/pods/67243d84-c09b-40d4-a6c1-aa2422713b76/volumes"
Jul 30 16:38:36.158783 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:36.158752    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:38:36.162430 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:36.162408    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-65495d8796-2pnkx"
Jul 30 16:38:58.065788 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:58.065748    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:38:58.083775 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:58.083749    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:38:58.760831 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:38:58.760807    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:11.816477 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:11.816397    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 16:39:11.817027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:11.816732    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="alertmanager" containerID="cri-o://a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10" gracePeriod=120
Jul 30 16:39:11.817027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:11.816790    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy-metric" containerID="cri-o://7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6" gracePeriod=120
Jul 30 16:39:11.817027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:11.816817    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy-web" containerID="cri-o://cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4" gracePeriod=120
Jul 30 16:39:11.817027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:11.816871    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="config-reloader" containerID="cri-o://11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8" gracePeriod=120
Jul 30 16:39:11.817027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:11.816891    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="prom-label-proxy" containerID="cri-o://ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955" gracePeriod=120
Jul 30 16:39:11.817027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:11.816991    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy" containerID="cri-o://e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9" gracePeriod=120
Jul 30 16:39:12.804276 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:12.804245    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerID="ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955" exitCode=0
Jul 30 16:39:12.804276 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:12.804273    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerID="e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9" exitCode=0
Jul 30 16:39:12.804428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:12.804285    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerID="11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8" exitCode=0
Jul 30 16:39:12.804428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:12.804301    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerID="a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10" exitCode=0
Jul 30 16:39:12.804428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:12.804276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerDied","Data":"ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955"}
Jul 30 16:39:12.804578 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:12.804436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerDied","Data":"e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9"}
Jul 30 16:39:12.804578 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:12.804490    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerDied","Data":"11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8"}
Jul 30 16:39:12.804578 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:12.804536    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerDied","Data":"a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10"}
Jul 30 16:39:13.050450 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.050426    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:39:13.155939 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.155912    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-cluster-tls-config\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.155939 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.155942    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-metrics-client-ca\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156130 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.155983    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-web\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156130 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156004    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-tls-assets\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156130 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156038    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-main-db\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156130 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156066    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156130 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156097    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-766sg\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-kube-api-access-766sg\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156130 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156123    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156167    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-web-config\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156207    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-main-tls\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156231    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-config-volume\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156270    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-config-out\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156302    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-trusted-ca-bundle\") pod \"4e19e9a0-b291-46dd-baaa-b347e892801f\" (UID: \"4e19e9a0-b291-46dd-baaa-b347e892801f\") "
Jul 30 16:39:13.156428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156398    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:39:13.156734 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156629    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:39:13.156794 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156729    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-main-db\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.156794 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156737    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:39:13.156794 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.156749    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-metrics-client-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.159241 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.159172    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-config-out" (OuterVolumeSpecName: "config-out") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:39:13.159241 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.159177    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:13.159444 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.159406    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:13.159566 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.159542    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:39:13.159640 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.159627    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-kube-api-access-766sg" (OuterVolumeSpecName: "kube-api-access-766sg") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "kube-api-access-766sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:39:13.159770 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.159751    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-config-volume" (OuterVolumeSpecName: "config-volume") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:13.159834 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.159771    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:13.160091 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.160073    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:13.163475 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.163331    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:13.168866 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.168844    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-web-config" (OuterVolumeSpecName: "web-config") pod "4e19e9a0-b291-46dd-baaa-b347e892801f" (UID: "4e19e9a0-b291-46dd-baaa-b347e892801f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:13.257592 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257571    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257592 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257591    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-tls-assets\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257719 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257601    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257719 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257626    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-766sg\" (UniqueName: \"kubernetes.io/projected/4e19e9a0-b291-46dd-baaa-b347e892801f-kube-api-access-766sg\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257719 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257635    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257719 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257643    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-web-config\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257719 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257651    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-secret-alertmanager-main-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257719 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257659    2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257719 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257666    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4e19e9a0-b291-46dd-baaa-b347e892801f-config-out\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257719 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257674    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e19e9a0-b291-46dd-baaa-b347e892801f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.257719 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.257681    2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4e19e9a0-b291-46dd-baaa-b347e892801f-cluster-tls-config\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:13.810322 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.810296    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerID="7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6" exitCode=0
Jul 30 16:39:13.810322 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.810318    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerID="cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4" exitCode=0
Jul 30 16:39:13.810527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.810382    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 16:39:13.810527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.810375    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerDied","Data":"7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6"}
Jul 30 16:39:13.810527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.810515    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerDied","Data":"cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4"}
Jul 30 16:39:13.810700 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.810544    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4e19e9a0-b291-46dd-baaa-b347e892801f","Type":"ContainerDied","Data":"4590e15c223e8920548d08f904b8de3e48ab6154e5d48815a06e9dc920b65569"}
Jul 30 16:39:13.810700 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.810575    2569 scope.go:117] "RemoveContainer" containerID="ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955"
Jul 30 16:39:13.817551 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.817519    2569 scope.go:117] "RemoveContainer" containerID="7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6"
Jul 30 16:39:13.824598 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.824554    2569 scope.go:117] "RemoveContainer" containerID="e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9"
Jul 30 16:39:13.830818 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.830802    2569 scope.go:117] "RemoveContainer" containerID="cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4"
Jul 30 16:39:13.836010 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.835979    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 16:39:13.837638 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.837623    2569 scope.go:117] "RemoveContainer" containerID="11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8"
Jul 30 16:39:13.842101 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.842083    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 16:39:13.844815 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.844800    2569 scope.go:117] "RemoveContainer" containerID="a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10"
Jul 30 16:39:13.851029 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.851005    2569 scope.go:117] "RemoveContainer" containerID="9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c"
Jul 30 16:39:13.857241 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.857221    2569 scope.go:117] "RemoveContainer" containerID="ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955"
Jul 30 16:39:13.857452 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:13.857434    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955\": container with ID starting with ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955 not found: ID does not exist" containerID="ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955"
Jul 30 16:39:13.857502 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.857458    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955"} err="failed to get container status \"ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955\": rpc error: code = NotFound desc = could not find container \"ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955\": container with ID starting with ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955 not found: ID does not exist"
Jul 30 16:39:13.857502 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.857472    2569 scope.go:117] "RemoveContainer" containerID="7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6"
Jul 30 16:39:13.857731 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:13.857703    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6\": container with ID starting with 7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6 not found: ID does not exist" containerID="7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6"
Jul 30 16:39:13.857805 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.857739    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6"} err="failed to get container status \"7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6\": rpc error: code = NotFound desc = could not find container \"7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6\": container with ID starting with 7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6 not found: ID does not exist"
Jul 30 16:39:13.857805 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.857760    2569 scope.go:117] "RemoveContainer" containerID="e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9"
Jul 30 16:39:13.857999 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:13.857984    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9\": container with ID starting with e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9 not found: ID does not exist" containerID="e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9"
Jul 30 16:39:13.858067 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858005    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9"} err="failed to get container status \"e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9\": rpc error: code = NotFound desc = could not find container \"e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9\": container with ID starting with e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9 not found: ID does not exist"
Jul 30 16:39:13.858067 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858025    2569 scope.go:117] "RemoveContainer" containerID="cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4"
Jul 30 16:39:13.858233 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:13.858218    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4\": container with ID starting with cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4 not found: ID does not exist" containerID="cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4"
Jul 30 16:39:13.858286 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858238    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4"} err="failed to get container status \"cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4\": rpc error: code = NotFound desc = could not find container \"cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4\": container with ID starting with cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4 not found: ID does not exist"
Jul 30 16:39:13.858286 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858256    2569 scope.go:117] "RemoveContainer" containerID="11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8"
Jul 30 16:39:13.858439 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:13.858423    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8\": container with ID starting with 11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8 not found: ID does not exist" containerID="11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8"
Jul 30 16:39:13.858497 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858444    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8"} err="failed to get container status \"11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8\": rpc error: code = NotFound desc = could not find container \"11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8\": container with ID starting with 11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8 not found: ID does not exist"
Jul 30 16:39:13.858497 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858460    2569 scope.go:117] "RemoveContainer" containerID="a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10"
Jul 30 16:39:13.858677 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:13.858659    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10\": container with ID starting with a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10 not found: ID does not exist" containerID="a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10"
Jul 30 16:39:13.858723 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858681    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10"} err="failed to get container status \"a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10\": rpc error: code = NotFound desc = could not find container \"a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10\": container with ID starting with a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10 not found: ID does not exist"
Jul 30 16:39:13.858723 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858695    2569 scope.go:117] "RemoveContainer" containerID="9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c"
Jul 30 16:39:13.858868 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:13.858853    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c\": container with ID starting with 9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c not found: ID does not exist" containerID="9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c"
Jul 30 16:39:13.858926 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858874    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c"} err="failed to get container status \"9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c\": rpc error: code = NotFound desc = could not find container \"9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c\": container with ID starting with 9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c not found: ID does not exist"
Jul 30 16:39:13.858926 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.858895    2569 scope.go:117] "RemoveContainer" containerID="ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955"
Jul 30 16:39:13.859108 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859090    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955"} err="failed to get container status \"ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955\": rpc error: code = NotFound desc = could not find container \"ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955\": container with ID starting with ed98ecd3c7f1005d4d7347a35d51165c6b04c83d5eddee029894908c45d44955 not found: ID does not exist"
Jul 30 16:39:13.859154 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859111    2569 scope.go:117] "RemoveContainer" containerID="7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6"
Jul 30 16:39:13.859301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859287    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6"} err="failed to get container status \"7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6\": rpc error: code = NotFound desc = could not find container \"7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6\": container with ID starting with 7471fe4be6060e1e1d267627d1a2962b2b9930b24d488d6f6b7ebab9fb7715c6 not found: ID does not exist"
Jul 30 16:39:13.859344 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859301    2569 scope.go:117] "RemoveContainer" containerID="e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9"
Jul 30 16:39:13.859513 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859495    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9"} err="failed to get container status \"e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9\": rpc error: code = NotFound desc = could not find container \"e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9\": container with ID starting with e569c06108b9bbf5c678590120667966917ed8ee397a3b406425f0eb19fedef9 not found: ID does not exist"
Jul 30 16:39:13.859584 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859513    2569 scope.go:117] "RemoveContainer" containerID="cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4"
Jul 30 16:39:13.859759 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859739    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4"} err="failed to get container status \"cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4\": rpc error: code = NotFound desc = could not find container \"cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4\": container with ID starting with cd008651cee1b0ac0c2ccbb3e40c0c9dcd3b8c54df96ecbf24f57c405ad514a4 not found: ID does not exist"
Jul 30 16:39:13.859759 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859758    2569 scope.go:117] "RemoveContainer" containerID="11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8"
Jul 30 16:39:13.859936 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859921    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8"} err="failed to get container status \"11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8\": rpc error: code = NotFound desc = could not find container \"11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8\": container with ID starting with 11524abbbf8fa3e37a684ca89008d76ba34294e5ac09dffa416fe63552bcf2b8 not found: ID does not exist"
Jul 30 16:39:13.859973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.859938    2569 scope.go:117] "RemoveContainer" containerID="a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10"
Jul 30 16:39:13.860143 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.860129    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10"} err="failed to get container status \"a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10\": rpc error: code = NotFound desc = could not find container \"a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10\": container with ID starting with a007876f24857d8d07343157a11fb1e241a9a2bff2f915bb655549fe9ea3cb10 not found: ID does not exist"
Jul 30 16:39:13.860185 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.860142    2569 scope.go:117] "RemoveContainer" containerID="9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c"
Jul 30 16:39:13.860334 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:13.860318    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c"} err="failed to get container status \"9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c\": rpc error: code = NotFound desc = could not find container \"9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c\": container with ID starting with 9f6157549fd2632bb613e3a9fd055c788f8c538a2de713979dc28a49e785cd5c not found: ID does not exist"
Jul 30 16:39:14.999271 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:14.999239    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" path="/var/lib/kubelet/pods/4e19e9a0-b291-46dd-baaa-b347e892801f/volumes"
Jul 30 16:39:16.111414 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.111372    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 16:39:16.112041 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.112006    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="prometheus" containerID="cri-o://8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb" gracePeriod=600
Jul 30 16:39:16.112041 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.112026    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy-web" containerID="cri-o://aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be" gracePeriod=600
Jul 30 16:39:16.112243 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.112007    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143" gracePeriod=600
Jul 30 16:39:16.112243 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.112154    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy" containerID="cri-o://b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4" gracePeriod=600
Jul 30 16:39:16.112243 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.112191    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="thanos-sidecar" containerID="cri-o://6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c" gracePeriod=600
Jul 30 16:39:16.112395 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.112297    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="config-reloader" containerID="cri-o://69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0" gracePeriod=600
Jul 30 16:39:16.355150 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.355127    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.480574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.480544    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-web-config\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.480574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.480577    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-db\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.480828 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.480620    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-2\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.480828 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.480637    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-kube-rbac-proxy\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.480828 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.480652    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-tls\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.480828 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.480675    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-thanos-prometheus-http-client-file\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.480915    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj4xs\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-kube-api-access-dj4xs\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.480938    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:39:16.481027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.480982    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-config-out\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481167 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481038    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-tls-assets\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481167 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481060    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-config\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481167 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481083    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-grpc-tls\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481167 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481112    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-metrics-client-ca\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481167 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481141    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-serving-certs-ca-bundle\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481408 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481176    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-kubelet-serving-ca-bundle\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481408 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481201    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-0\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481408 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481242    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-1\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481408 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481274    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481408 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481302    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481408 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481327    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-trusted-ca-bundle\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481408 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481357    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-metrics-client-certs\") pod \"d0ea597c-c679-4fb9-810a-44953e63eb14\" (UID: \"d0ea597c-c679-4fb9-810a-44953e63eb14\") "
Jul 30 16:39:16.481749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481594    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.481749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.481695    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:39:16.482632 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.482022    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:39:16.482632 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.482334    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:39:16.482632 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.482528    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:39:16.482855 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.482773    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:39:16.483736 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.483548    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:16.483736 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.483632    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:16.483736 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.483702    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-kube-api-access-dj4xs" (OuterVolumeSpecName: "kube-api-access-dj4xs") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "kube-api-access-dj4xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:39:16.483937 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.483851    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:39:16.484246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.484222    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:39:16.484980 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.484954    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:39:16.485088 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.485067    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:16.485146 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.485086    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-config-out" (OuterVolumeSpecName: "config-out") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 16:39:16.485146 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.485107    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-config" (OuterVolumeSpecName: "config") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:16.485252 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.485222    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:16.485917 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.485883    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:16.486351 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.486326    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:16.486421 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.486386    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:16.493156 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.493138    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-web-config" (OuterVolumeSpecName: "web-config") pod "d0ea597c-c679-4fb9-810a-44953e63eb14" (UID: "d0ea597c-c679-4fb9-810a-44953e63eb14"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:39:16.582840 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582816    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.582942 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582840    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.582942 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582855    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.582942 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582869    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.582942 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582881    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-metrics-client-certs\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.582942 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582893    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-web-config\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.582942 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582906    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-db\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.582942 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582918    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-kube-rbac-proxy\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.582942 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582930    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.582942 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582943    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.583246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582956    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj4xs\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-kube-api-access-dj4xs\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.583246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582968    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d0ea597c-c679-4fb9-810a-44953e63eb14-config-out\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.583246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582981    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d0ea597c-c679-4fb9-810a-44953e63eb14-tls-assets\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.583246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.582993    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-config\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.583246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.583004    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d0ea597c-c679-4fb9-810a-44953e63eb14-secret-grpc-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.583246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.583016    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-metrics-client-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.583246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.583029    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.583246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.583042    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.583246 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.583056    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d0ea597c-c679-4fb9-810a-44953e63eb14-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:39:16.824067 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824009    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerID="6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143" exitCode=0
Jul 30 16:39:16.824067 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824029    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerID="b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4" exitCode=0
Jul 30 16:39:16.824067 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824035    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerID="aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be" exitCode=0
Jul 30 16:39:16.824067 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824041    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerID="6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c" exitCode=0
Jul 30 16:39:16.824067 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824046    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerID="69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0" exitCode=0
Jul 30 16:39:16.824067 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824051    2569 generic.go:356] "Generic (PLEG): container finished" podID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerID="8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb" exitCode=0
Jul 30 16:39:16.824400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824075    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerDied","Data":"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"}
Jul 30 16:39:16.824400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824096    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.824400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824127    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerDied","Data":"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"}
Jul 30 16:39:16.824400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824153    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerDied","Data":"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"}
Jul 30 16:39:16.824400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824176    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerDied","Data":"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"}
Jul 30 16:39:16.824400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824199    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerDied","Data":"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"}
Jul 30 16:39:16.824400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824203    2569 scope.go:117] "RemoveContainer" containerID="6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"
Jul 30 16:39:16.824400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824223    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerDied","Data":"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"}
Jul 30 16:39:16.824400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.824246    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d0ea597c-c679-4fb9-810a-44953e63eb14","Type":"ContainerDied","Data":"a904a3ec1e546e8e769faf27ebdd3a24fff0daa01f2e6b2812bd8dbb9445584f"}
Jul 30 16:39:16.834149 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.834049    2569 scope.go:117] "RemoveContainer" containerID="b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"
Jul 30 16:39:16.840678 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.840663    2569 scope.go:117] "RemoveContainer" containerID="aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"
Jul 30 16:39:16.846847 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.846830    2569 scope.go:117] "RemoveContainer" containerID="6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"
Jul 30 16:39:16.851101 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.851072    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 16:39:16.854051 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.854028    2569 scope.go:117] "RemoveContainer" containerID="69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"
Jul 30 16:39:16.856220 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.856197    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 16:39:16.860357 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.860342    2569 scope.go:117] "RemoveContainer" containerID="8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"
Jul 30 16:39:16.866843 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.866824    2569 scope.go:117] "RemoveContainer" containerID="66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"
Jul 30 16:39:16.872961 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.872944    2569 scope.go:117] "RemoveContainer" containerID="6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"
Jul 30 16:39:16.873192 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:16.873164    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": container with ID starting with 6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143 not found: ID does not exist" containerID="6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"
Jul 30 16:39:16.873274 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.873201    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"} err="failed to get container status \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": rpc error: code = NotFound desc = could not find container \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": container with ID starting with 6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143 not found: ID does not exist"
Jul 30 16:39:16.873274 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.873222    2569 scope.go:117] "RemoveContainer" containerID="b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"
Jul 30 16:39:16.873454 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:16.873441    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": container with ID starting with b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4 not found: ID does not exist" containerID="b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"
Jul 30 16:39:16.873514 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.873462    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"} err="failed to get container status \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": rpc error: code = NotFound desc = could not find container \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": container with ID starting with b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4 not found: ID does not exist"
Jul 30 16:39:16.873514 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.873483    2569 scope.go:117] "RemoveContainer" containerID="aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"
Jul 30 16:39:16.873697 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:16.873676    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": container with ID starting with aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be not found: ID does not exist" containerID="aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"
Jul 30 16:39:16.873745 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.873703    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"} err="failed to get container status \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": rpc error: code = NotFound desc = could not find container \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": container with ID starting with aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be not found: ID does not exist"
Jul 30 16:39:16.873745 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.873720    2569 scope.go:117] "RemoveContainer" containerID="6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"
Jul 30 16:39:16.873929 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:16.873912    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": container with ID starting with 6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c not found: ID does not exist" containerID="6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"
Jul 30 16:39:16.873969 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.873934    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"} err="failed to get container status \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": rpc error: code = NotFound desc = could not find container \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": container with ID starting with 6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c not found: ID does not exist"
Jul 30 16:39:16.873969 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.873950    2569 scope.go:117] "RemoveContainer" containerID="69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"
Jul 30 16:39:16.874156 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:16.874140    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": container with ID starting with 69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0 not found: ID does not exist" containerID="69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"
Jul 30 16:39:16.874211 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.874163    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"} err="failed to get container status \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": rpc error: code = NotFound desc = could not find container \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": container with ID starting with 69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0 not found: ID does not exist"
Jul 30 16:39:16.874211 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.874185    2569 scope.go:117] "RemoveContainer" containerID="8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"
Jul 30 16:39:16.874422 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:16.874407    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": container with ID starting with 8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb not found: ID does not exist" containerID="8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"
Jul 30 16:39:16.874459 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.874427    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"} err="failed to get container status \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": rpc error: code = NotFound desc = could not find container \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": container with ID starting with 8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb not found: ID does not exist"
Jul 30 16:39:16.874459 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.874440    2569 scope.go:117] "RemoveContainer" containerID="66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"
Jul 30 16:39:16.874669 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:39:16.874654    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": container with ID starting with 66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340 not found: ID does not exist" containerID="66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"
Jul 30 16:39:16.874717 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.874672    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"} err="failed to get container status \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": rpc error: code = NotFound desc = could not find container \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": container with ID starting with 66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340 not found: ID does not exist"
Jul 30 16:39:16.874717 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.874685    2569 scope.go:117] "RemoveContainer" containerID="6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"
Jul 30 16:39:16.874907 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.874888    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"} err="failed to get container status \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": rpc error: code = NotFound desc = could not find container \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": container with ID starting with 6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143 not found: ID does not exist"
Jul 30 16:39:16.874971 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.874908    2569 scope.go:117] "RemoveContainer" containerID="b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"
Jul 30 16:39:16.875135 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875118    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"} err="failed to get container status \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": rpc error: code = NotFound desc = could not find container \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": container with ID starting with b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4 not found: ID does not exist"
Jul 30 16:39:16.875175 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875136    2569 scope.go:117] "RemoveContainer" containerID="aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"
Jul 30 16:39:16.875342 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875326    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"} err="failed to get container status \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": rpc error: code = NotFound desc = could not find container \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": container with ID starting with aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be not found: ID does not exist"
Jul 30 16:39:16.875386 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875342    2569 scope.go:117] "RemoveContainer" containerID="6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"
Jul 30 16:39:16.875530 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875513    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"} err="failed to get container status \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": rpc error: code = NotFound desc = could not find container \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": container with ID starting with 6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c not found: ID does not exist"
Jul 30 16:39:16.875573 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875529    2569 scope.go:117] "RemoveContainer" containerID="69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"
Jul 30 16:39:16.875717 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875702    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"} err="failed to get container status \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": rpc error: code = NotFound desc = could not find container \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": container with ID starting with 69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0 not found: ID does not exist"
Jul 30 16:39:16.875765 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875717    2569 scope.go:117] "RemoveContainer" containerID="8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"
Jul 30 16:39:16.875906 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875889    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"} err="failed to get container status \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": rpc error: code = NotFound desc = could not find container \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": container with ID starting with 8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb not found: ID does not exist"
Jul 30 16:39:16.875967 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.875907    2569 scope.go:117] "RemoveContainer" containerID="66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"
Jul 30 16:39:16.876141 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876123    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"} err="failed to get container status \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": rpc error: code = NotFound desc = could not find container \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": container with ID starting with 66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340 not found: ID does not exist"
Jul 30 16:39:16.876182 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876142    2569 scope.go:117] "RemoveContainer" containerID="6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"
Jul 30 16:39:16.876339 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876323    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"} err="failed to get container status \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": rpc error: code = NotFound desc = could not find container \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": container with ID starting with 6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143 not found: ID does not exist"
Jul 30 16:39:16.876395 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876340    2569 scope.go:117] "RemoveContainer" containerID="b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"
Jul 30 16:39:16.876566 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876548    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"} err="failed to get container status \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": rpc error: code = NotFound desc = could not find container \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": container with ID starting with b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4 not found: ID does not exist"
Jul 30 16:39:16.876644 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876567    2569 scope.go:117] "RemoveContainer" containerID="aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"
Jul 30 16:39:16.876789 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876771    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"} err="failed to get container status \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": rpc error: code = NotFound desc = could not find container \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": container with ID starting with aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be not found: ID does not exist"
Jul 30 16:39:16.876832 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876790    2569 scope.go:117] "RemoveContainer" containerID="6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"
Jul 30 16:39:16.876973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876957    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"} err="failed to get container status \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": rpc error: code = NotFound desc = could not find container \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": container with ID starting with 6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c not found: ID does not exist"
Jul 30 16:39:16.877040 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.876975    2569 scope.go:117] "RemoveContainer" containerID="69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"
Jul 30 16:39:16.877188 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.877170    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"} err="failed to get container status \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": rpc error: code = NotFound desc = could not find container \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": container with ID starting with 69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0 not found: ID does not exist"
Jul 30 16:39:16.877231 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.877190    2569 scope.go:117] "RemoveContainer" containerID="8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"
Jul 30 16:39:16.877402 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.877379    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"} err="failed to get container status \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": rpc error: code = NotFound desc = could not find container \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": container with ID starting with 8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb not found: ID does not exist"
Jul 30 16:39:16.877402 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.877400    2569 scope.go:117] "RemoveContainer" containerID="66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"
Jul 30 16:39:16.877642 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.877619    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"} err="failed to get container status \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": rpc error: code = NotFound desc = could not find container \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": container with ID starting with 66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340 not found: ID does not exist"
Jul 30 16:39:16.877720 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.877643    2569 scope.go:117] "RemoveContainer" containerID="6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"
Jul 30 16:39:16.877926 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.877847    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"} err="failed to get container status \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": rpc error: code = NotFound desc = could not find container \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": container with ID starting with 6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143 not found: ID does not exist"
Jul 30 16:39:16.877926 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.877870    2569 scope.go:117] "RemoveContainer" containerID="b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"
Jul 30 16:39:16.878109 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878091    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"} err="failed to get container status \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": rpc error: code = NotFound desc = could not find container \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": container with ID starting with b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4 not found: ID does not exist"
Jul 30 16:39:16.878174 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878110    2569 scope.go:117] "RemoveContainer" containerID="aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"
Jul 30 16:39:16.878354 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878334    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"} err="failed to get container status \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": rpc error: code = NotFound desc = could not find container \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": container with ID starting with aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be not found: ID does not exist"
Jul 30 16:39:16.878354 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878352    2569 scope.go:117] "RemoveContainer" containerID="6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"
Jul 30 16:39:16.878535 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878519    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"} err="failed to get container status \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": rpc error: code = NotFound desc = could not find container \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": container with ID starting with 6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c not found: ID does not exist"
Jul 30 16:39:16.878577 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878559    2569 scope.go:117] "RemoveContainer" containerID="69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"
Jul 30 16:39:16.878762 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878747    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"} err="failed to get container status \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": rpc error: code = NotFound desc = could not find container \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": container with ID starting with 69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0 not found: ID does not exist"
Jul 30 16:39:16.878804 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878762    2569 scope.go:117] "RemoveContainer" containerID="8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"
Jul 30 16:39:16.878958 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878942    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"} err="failed to get container status \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": rpc error: code = NotFound desc = could not find container \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": container with ID starting with 8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb not found: ID does not exist"
Jul 30 16:39:16.879004 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.878958    2569 scope.go:117] "RemoveContainer" containerID="66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"
Jul 30 16:39:16.879127 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879112    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"} err="failed to get container status \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": rpc error: code = NotFound desc = could not find container \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": container with ID starting with 66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340 not found: ID does not exist"
Jul 30 16:39:16.879168 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879127    2569 scope.go:117] "RemoveContainer" containerID="6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"
Jul 30 16:39:16.879304 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879283    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"} err="failed to get container status \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": rpc error: code = NotFound desc = could not find container \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": container with ID starting with 6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143 not found: ID does not exist"
Jul 30 16:39:16.879342 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879306    2569 scope.go:117] "RemoveContainer" containerID="b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"
Jul 30 16:39:16.879461 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879446    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"} err="failed to get container status \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": rpc error: code = NotFound desc = could not find container \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": container with ID starting with b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4 not found: ID does not exist"
Jul 30 16:39:16.879506 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879460    2569 scope.go:117] "RemoveContainer" containerID="aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"
Jul 30 16:39:16.879687 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879671    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"} err="failed to get container status \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": rpc error: code = NotFound desc = could not find container \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": container with ID starting with aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be not found: ID does not exist"
Jul 30 16:39:16.879725 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879688    2569 scope.go:117] "RemoveContainer" containerID="6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"
Jul 30 16:39:16.879877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879859    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"} err="failed to get container status \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": rpc error: code = NotFound desc = could not find container \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": container with ID starting with 6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c not found: ID does not exist"
Jul 30 16:39:16.879920 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.879878    2569 scope.go:117] "RemoveContainer" containerID="69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"
Jul 30 16:39:16.880076 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880061    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"} err="failed to get container status \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": rpc error: code = NotFound desc = could not find container \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": container with ID starting with 69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0 not found: ID does not exist"
Jul 30 16:39:16.880076 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880075    2569 scope.go:117] "RemoveContainer" containerID="8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"
Jul 30 16:39:16.880249 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880231    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"} err="failed to get container status \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": rpc error: code = NotFound desc = could not find container \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": container with ID starting with 8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb not found: ID does not exist"
Jul 30 16:39:16.880290 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880249    2569 scope.go:117] "RemoveContainer" containerID="66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"
Jul 30 16:39:16.880442 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880427    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"} err="failed to get container status \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": rpc error: code = NotFound desc = could not find container \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": container with ID starting with 66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340 not found: ID does not exist"
Jul 30 16:39:16.880442 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880442    2569 scope.go:117] "RemoveContainer" containerID="6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"
Jul 30 16:39:16.880647 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880631    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143"} err="failed to get container status \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": rpc error: code = NotFound desc = could not find container \"6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143\": container with ID starting with 6cb5cee6237ca1d67abf7cf20c356ef28daf35a31f8ab6350a3fdad095b9f143 not found: ID does not exist"
Jul 30 16:39:16.880690 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880646    2569 scope.go:117] "RemoveContainer" containerID="b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"
Jul 30 16:39:16.880846 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880830    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4"} err="failed to get container status \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": rpc error: code = NotFound desc = could not find container \"b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4\": container with ID starting with b3d855c2cc205a66eb072e66b589993777ace63c38845782dee127e5261133c4 not found: ID does not exist"
Jul 30 16:39:16.880889 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.880846    2569 scope.go:117] "RemoveContainer" containerID="aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"
Jul 30 16:39:16.881048 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.881033    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be"} err="failed to get container status \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": rpc error: code = NotFound desc = could not find container \"aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be\": container with ID starting with aaaaf52fd42d6cd916580f5087143e99dbd989361b83c4e6aee598509668b6be not found: ID does not exist"
Jul 30 16:39:16.881101 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.881049    2569 scope.go:117] "RemoveContainer" containerID="6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"
Jul 30 16:39:16.881255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.881236    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c"} err="failed to get container status \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": rpc error: code = NotFound desc = could not find container \"6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c\": container with ID starting with 6855f211649b18fce5d68a4d54b6b1496e160e8c5db96959c272730f96ce007c not found: ID does not exist"
Jul 30 16:39:16.881310 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.881256    2569 scope.go:117] "RemoveContainer" containerID="69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"
Jul 30 16:39:16.881456 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.881441    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0"} err="failed to get container status \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": rpc error: code = NotFound desc = could not find container \"69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0\": container with ID starting with 69159e4a3c308af170e8114b0031389d154c3c52ccd6497d731bec12d36d86b0 not found: ID does not exist"
Jul 30 16:39:16.881492 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.881457    2569 scope.go:117] "RemoveContainer" containerID="8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"
Jul 30 16:39:16.881676 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.881660    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb"} err="failed to get container status \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": rpc error: code = NotFound desc = could not find container \"8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb\": container with ID starting with 8a3f73d6ab4784e416db596465f551e88d107a6043673d819d902b2719f2decb not found: ID does not exist"
Jul 30 16:39:16.881676 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.881676    2569 scope.go:117] "RemoveContainer" containerID="66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"
Jul 30 16:39:16.881894 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.881876    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340"} err="failed to get container status \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": rpc error: code = NotFound desc = could not find container \"66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340\": container with ID starting with 66abeaaae567f41bb0bd193b59f7db6dd6c8320ba6f70cf8c0c325d713b64340 not found: ID does not exist"
Jul 30 16:39:16.887401 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887381    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 16:39:16.887643 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887631    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy"
Jul 30 16:39:16.887685 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887644    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy"
Jul 30 16:39:16.887685 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887652    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy-metric"
Jul 30 16:39:16.887685 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887657    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy-metric"
Jul 30 16:39:16.887685 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887664    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="init-config-reloader"
Jul 30 16:39:16.887685 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887670    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="init-config-reloader"
Jul 30 16:39:16.887685 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887676    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy-web"
Jul 30 16:39:16.887685 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887681    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy-web"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887692    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="init-config-reloader"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887697    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="init-config-reloader"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887703    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy-web"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887708    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy-web"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887715    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="thanos-sidecar"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887720    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="thanos-sidecar"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887726    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy-thanos"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887731    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy-thanos"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887737    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="config-reloader"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887742    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="config-reloader"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887751    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="alertmanager"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887755    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="alertmanager"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887761    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67243d84-c09b-40d4-a6c1-aa2422713b76" containerName="console"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887766    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="67243d84-c09b-40d4-a6c1-aa2422713b76" containerName="console"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887772    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="prometheus"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887776    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="prometheus"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887781    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887786    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887792    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="config-reloader"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887796    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="config-reloader"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887803    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="prom-label-proxy"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887808    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="prom-label-proxy"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887845    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="67243d84-c09b-40d4-a6c1-aa2422713b76" containerName="console"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887851    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="config-reloader"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887858    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="prometheus"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887863    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="config-reloader"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887869    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="alertmanager"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887875    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="prom-label-proxy"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887880    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy"
Jul 30 16:39:16.887877 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887886    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy-web"
Jul 30 16:39:16.888674 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887892    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="thanos-sidecar"
Jul 30 16:39:16.888674 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887897    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy"
Jul 30 16:39:16.888674 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887903    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e19e9a0-b291-46dd-baaa-b347e892801f" containerName="kube-rbac-proxy-metric"
Jul 30 16:39:16.888674 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887908    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy-web"
Jul 30 16:39:16.888674 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.887912    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" containerName="kube-rbac-proxy-thanos"
Jul 30 16:39:16.891071 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.891054    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.894208 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.894185    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 30 16:39:16.894301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.894247    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 30 16:39:16.894999 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.894980    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 30 16:39:16.895245 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.895228    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 30 16:39:16.895333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.895228    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 30 16:39:16.895333 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.895291    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4oggtuj18a2q0\""
Jul 30 16:39:16.895676 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.895655    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 30 16:39:16.895826 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.895814    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 30 16:39:16.895938 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.895858    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bxw24\""
Jul 30 16:39:16.895938 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.895898    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 30 16:39:16.895938 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.895930    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 30 16:39:16.896239 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.896143    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 30 16:39:16.896733 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.896714    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 30 16:39:16.896813 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.896736    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 30 16:39:16.902379 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.902349    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 30 16:39:16.905383 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.905355    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 30 16:39:16.914073 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.913959    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 16:39:16.985859 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.985826    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.985859 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.985865    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5be07d24-4225-4e66-abf4-2cf4536dfa4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.985985 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.985885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.985985 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.985907    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.985985 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.985930    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.985985 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.985950    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.985985 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.985965    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.985985 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.985980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986153 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986025    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986153 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986061    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-config\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986153 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986082    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986153 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986109    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986153 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986296 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986159    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj89f\" (UniqueName: \"kubernetes.io/projected/5be07d24-4225-4e66-abf4-2cf4536dfa4c-kube-api-access-dj89f\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986296 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986179    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5be07d24-4225-4e66-abf4-2cf4536dfa4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986296 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986195    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986296 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986246    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986296 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986280    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986432 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986300    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.986432 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.986320    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:16.999337 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:16.999313    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ea597c-c679-4fb9-810a-44953e63eb14" path="/var/lib/kubelet/pods/d0ea597c-c679-4fb9-810a-44953e63eb14/volumes"
Jul 30 16:39:17.087021 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.086970    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087021 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087027    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087055    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087078    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5be07d24-4225-4e66-abf4-2cf4536dfa4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087130    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087179    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087204    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087229    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087283    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-config\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087375    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087437 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087400    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj89f\" (UniqueName: \"kubernetes.io/projected/5be07d24-4225-4e66-abf4-2cf4536dfa4c-kube-api-access-dj89f\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087447    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5be07d24-4225-4e66-abf4-2cf4536dfa4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087471    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087907    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087907    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.087973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.087951    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.088592 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.088494    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.090176 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.089705    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.090176 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.089890    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.090176 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.090139    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.090535 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.090509    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.090835 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.090814    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.091518 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.091104    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5be07d24-4225-4e66-abf4-2cf4536dfa4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.091518 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.091265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.091518 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.091471    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.091737 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.091661    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-config\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.091838 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.091820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.091971 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.091952    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.092008 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.091969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.092351 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.092334    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5be07d24-4225-4e66-abf4-2cf4536dfa4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.093101 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.093086    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5be07d24-4225-4e66-abf4-2cf4536dfa4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.093148 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.093119    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5be07d24-4225-4e66-abf4-2cf4536dfa4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.097879 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.097863    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj89f\" (UniqueName: \"kubernetes.io/projected/5be07d24-4225-4e66-abf4-2cf4536dfa4c-kube-api-access-dj89f\") pod \"prometheus-k8s-0\" (UID: \"5be07d24-4225-4e66-abf4-2cf4536dfa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.205245 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.205225    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:39:17.356368 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.356307    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 30 16:39:17.359585 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:39:17.359560    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be07d24_4225_4e66_abf4_2cf4536dfa4c.slice/crio-76447fe52127fb33c3ab88fef588b1fed4b121bef1d34b4ab51060644a4527b3 WatchSource:0}: Error finding container 76447fe52127fb33c3ab88fef588b1fed4b121bef1d34b4ab51060644a4527b3: Status 404 returned error can't find the container with id 76447fe52127fb33c3ab88fef588b1fed4b121bef1d34b4ab51060644a4527b3
Jul 30 16:39:17.830670 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.830643    2569 generic.go:356] "Generic (PLEG): container finished" podID="5be07d24-4225-4e66-abf4-2cf4536dfa4c" containerID="4acda3f64052e494cd4a90b32f689bf5d42853f6a879a6b081516b13cf49b328" exitCode=0
Jul 30 16:39:17.830788 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.830672    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5be07d24-4225-4e66-abf4-2cf4536dfa4c","Type":"ContainerDied","Data":"4acda3f64052e494cd4a90b32f689bf5d42853f6a879a6b081516b13cf49b328"}
Jul 30 16:39:17.830788 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:17.830732    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5be07d24-4225-4e66-abf4-2cf4536dfa4c","Type":"ContainerStarted","Data":"76447fe52127fb33c3ab88fef588b1fed4b121bef1d34b4ab51060644a4527b3"}
Jul 30 16:39:18.841904 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:18.841873    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5be07d24-4225-4e66-abf4-2cf4536dfa4c","Type":"ContainerStarted","Data":"96be1c30ea17d2a2cd38d21bf4c57416e06e6438eb015c1c5ab8580dcd5b46d2"}
Jul 30 16:39:18.842367 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:18.841976    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5be07d24-4225-4e66-abf4-2cf4536dfa4c","Type":"ContainerStarted","Data":"4299a1dd87397f10089eac17249950a721c7ab3e8ac9ec3c17fd46b13bcbb771"}
Jul 30 16:39:18.842367 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:18.842020    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5be07d24-4225-4e66-abf4-2cf4536dfa4c","Type":"ContainerStarted","Data":"6e6b7faee675a823235dcf6db4b7f2a554e550e9d44efab6acdd03903c8b869b"}
Jul 30 16:39:18.842367 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:18.842069    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5be07d24-4225-4e66-abf4-2cf4536dfa4c","Type":"ContainerStarted","Data":"347818ad801da5f7bb7cdf55fc82767c8f81fbf5e31019b7779c18b9e4dd09a2"}
Jul 30 16:39:18.842367 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:18.842104    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5be07d24-4225-4e66-abf4-2cf4536dfa4c","Type":"ContainerStarted","Data":"1d2c8a4f11b0c172b549ff327180116c4bbdd87680bace8a11198fddf407c36e"}
Jul 30 16:39:18.842367 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:18.842149    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5be07d24-4225-4e66-abf4-2cf4536dfa4c","Type":"ContainerStarted","Data":"f6684f4c94c909386559e4be73f1949d066ba608f9815e120568c27c9b9a2997"}
Jul 30 16:39:18.876086 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:18.876038    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.863486359 podStartE2EDuration="2.876025111s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.538752ms" isStatefulPod=false podCreationTimestamp="2026-07-30 16:39:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:39:18.874094722 +0000 UTC m=+186.476522346" watchObservedRunningTime="2026-07-30 16:39:18.876025111 +0000 UTC m=+186.478452734"
Jul 30 16:39:22.205674 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:39:22.205646    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:40:17.206290 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:40:17.206255    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:40:17.220730 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:40:17.220706    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:40:18.061644 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:40:18.061600    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 30 16:41:12.805027 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:41:12.804996    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 16:42:46.926622 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.926584    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"]
Jul 30 16:42:46.929560 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.929539    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:46.933962 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.933935    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 16:42:46.934073 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.933937    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 16:42:46.934073 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.933994    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 30 16:42:46.934818 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.934796    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 30 16:42:46.934941 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.934839    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 30 16:42:46.934941 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.934846    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-862b7\""
Jul 30 16:42:46.947439 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.947416    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"]
Jul 30 16:42:46.949383 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.949363    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1bba25f-b4f4-4130-9822-96c0b8d19408-metrics-cert\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:46.949490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.949406    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdrm6\" (UniqueName: \"kubernetes.io/projected/b1bba25f-b4f4-4130-9822-96c0b8d19408-kube-api-access-gdrm6\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:46.949490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.949434    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1bba25f-b4f4-4130-9822-96c0b8d19408-cert\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:46.949490 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:46.949477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b1bba25f-b4f4-4130-9822-96c0b8d19408-manager-config\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.050777 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.050750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b1bba25f-b4f4-4130-9822-96c0b8d19408-manager-config\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.050953 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.050798    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1bba25f-b4f4-4130-9822-96c0b8d19408-metrics-cert\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.051048 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.051027    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdrm6\" (UniqueName: \"kubernetes.io/projected/b1bba25f-b4f4-4130-9822-96c0b8d19408-kube-api-access-gdrm6\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.051109 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.051075    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1bba25f-b4f4-4130-9822-96c0b8d19408-cert\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.051545 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.051524    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b1bba25f-b4f4-4130-9822-96c0b8d19408-manager-config\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.053668 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.053601    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1bba25f-b4f4-4130-9822-96c0b8d19408-cert\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.057579 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.057557    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1bba25f-b4f4-4130-9822-96c0b8d19408-metrics-cert\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.098521 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.098492    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdrm6\" (UniqueName: \"kubernetes.io/projected/b1bba25f-b4f4-4130-9822-96c0b8d19408-kube-api-access-gdrm6\") pod \"lws-controller-manager-5b8dffc46b-98kjc\" (UID: \"b1bba25f-b4f4-4130-9822-96c0b8d19408\") " pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.238627 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.238539    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:47.382075 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.382047    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"]
Jul 30 16:42:47.385358 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:42:47.385328    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1bba25f_b4f4_4130_9822_96c0b8d19408.slice/crio-b5483adde2afa0a51d572b185de6742e7b65ba5fbb026d2aec9869e34189609b WatchSource:0}: Error finding container b5483adde2afa0a51d572b185de6742e7b65ba5fbb026d2aec9869e34189609b: Status 404 returned error can't find the container with id b5483adde2afa0a51d572b185de6742e7b65ba5fbb026d2aec9869e34189609b
Jul 30 16:42:47.387311 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.387294    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 16:42:47.566423 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:47.566350    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc" event={"ID":"b1bba25f-b4f4-4130-9822-96c0b8d19408","Type":"ContainerStarted","Data":"b5483adde2afa0a51d572b185de6742e7b65ba5fbb026d2aec9869e34189609b"}
Jul 30 16:42:48.519023 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.518987    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"]
Jul 30 16:42:48.523309 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.523291    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.539276 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.539258    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lp2wm\""
Jul 30 16:42:48.552653 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.552632    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 16:42:48.552762 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.552679    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 16:42:48.552762 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.552742    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 16:42:48.552895 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.552880    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 16:42:48.564665 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.564642    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.564762 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.564678    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw7v5\" (UniqueName: \"kubernetes.io/projected/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-kube-api-access-vw7v5\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.564762 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.564705    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.564863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.564761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-tmp\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.564863 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.564799    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-manifests\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.572400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.572372    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"]
Jul 30 16:42:48.665631 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.665596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-tmp\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.665767 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.665661    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-manifests\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.665767 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.665745    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.665886 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.665775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw7v5\" (UniqueName: \"kubernetes.io/projected/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-kube-api-access-vw7v5\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.665886 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.665809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.666064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.666042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-manifests\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.668304 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.668281    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-tmp\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.668431 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.668400    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.668511 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.668460    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.678453 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.678430    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw7v5\" (UniqueName: \"kubernetes.io/projected/238cc14e-586d-4938-8b5a-c9f2d0cce2e8-kube-api-access-vw7v5\") pod \"opendatahub-operator-controller-manager-7bfff65bb6-dxns8\" (UID: \"238cc14e-586d-4938-8b5a-c9f2d0cce2e8\") " pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.833622 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.833530    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:48.994520 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:48.994492    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"]
Jul 30 16:42:49.002565 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:42:49.002534    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod238cc14e_586d_4938_8b5a_c9f2d0cce2e8.slice/crio-dce94c60ae50dcfb9a0a768b1c5d91f7484774206c0dd6be081f1d83e604a0bc WatchSource:0}: Error finding container dce94c60ae50dcfb9a0a768b1c5d91f7484774206c0dd6be081f1d83e604a0bc: Status 404 returned error can't find the container with id dce94c60ae50dcfb9a0a768b1c5d91f7484774206c0dd6be081f1d83e604a0bc
Jul 30 16:42:49.575217 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:49.575183    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8" event={"ID":"238cc14e-586d-4938-8b5a-c9f2d0cce2e8","Type":"ContainerStarted","Data":"dce94c60ae50dcfb9a0a768b1c5d91f7484774206c0dd6be081f1d83e604a0bc"}
Jul 30 16:42:50.582645 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:50.582220    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc" event={"ID":"b1bba25f-b4f4-4130-9822-96c0b8d19408","Type":"ContainerStarted","Data":"6eb48283d9c34e15774fe98228603b4f605d91f4d10db8c456868d9051bc22aa"}
Jul 30 16:42:50.582645 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:50.582371    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:42:50.601736 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:50.601652    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc" podStartSLOduration=1.709410423 podStartE2EDuration="4.601635313s" totalImagesPullingTime="2.89222489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:42:50.601252256 +0000 UTC m=+398.203679884" watchObservedRunningTime="2026-07-30 16:42:50.601635313 +0000 UTC m=+398.204062940"
Jul 30 16:42:53.596652 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:53.596598    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8" event={"ID":"238cc14e-586d-4938-8b5a-c9f2d0cce2e8","Type":"ContainerStarted","Data":"8f9e2c928af70c18b992e5c8de0d6608407fbdb8eca98ad489a94481f64699dc"}
Jul 30 16:42:54.603006 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:54.602965    2569 generic.go:356] "Generic (PLEG): container finished" podID="238cc14e-586d-4938-8b5a-c9f2d0cce2e8" containerID="8f9e2c928af70c18b992e5c8de0d6608407fbdb8eca98ad489a94481f64699dc" exitCode=0
Jul 30 16:42:54.603531 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:54.603015    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8" event={"ID":"238cc14e-586d-4938-8b5a-c9f2d0cce2e8","Type":"ContainerDied","Data":"8f9e2c928af70c18b992e5c8de0d6608407fbdb8eca98ad489a94481f64699dc"}
Jul 30 16:42:55.609637 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:55.609576    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8" event={"ID":"238cc14e-586d-4938-8b5a-c9f2d0cce2e8","Type":"ContainerStarted","Data":"d1eab50897c7052a529e04e84250ead9dde1af3f086440546784ca572c506ed7"}
Jul 30 16:42:55.610112 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:55.609737    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:42:55.634205 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:42:55.634154    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8" podStartSLOduration=2.558726703 podStartE2EDuration="7.634140123s" totalImagesPullingTime="4.047293315s" totalInitContainerRuntime="1.028120105s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:42:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:42:55.632412796 +0000 UTC m=+403.234840427" watchObservedRunningTime="2026-07-30 16:42:55.634140123 +0000 UTC m=+403.236567747"
Jul 30 16:43:01.589715 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:01.589677    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b8dffc46b-98kjc"
Jul 30 16:43:06.615139 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:06.615101    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bfff65bb6-dxns8"
Jul 30 16:43:13.974403 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:13.974364    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"]
Jul 30 16:43:13.986569 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:13.986543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:13.990391 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:13.990367    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 16:43:13.990559 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:13.990408    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 16:43:13.991577 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:13.991558    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 16:43:13.991693 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:13.991595    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 16:43:13.991693 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:13.991559    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qdc4m\""
Jul 30 16:43:13.994053 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:13.993758    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"]
Jul 30 16:43:14.043049 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.043026    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d99c44c-128d-4fe8-8d3d-a8d655e18cd9-tls-certs\") pod \"kube-auth-proxy-bbc57f5f6-wbpzm\" (UID: \"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.043136 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.043059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgb6d\" (UniqueName: \"kubernetes.io/projected/9d99c44c-128d-4fe8-8d3d-a8d655e18cd9-kube-api-access-dgb6d\") pod \"kube-auth-proxy-bbc57f5f6-wbpzm\" (UID: \"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.043136 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.043084    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d99c44c-128d-4fe8-8d3d-a8d655e18cd9-tmp\") pod \"kube-auth-proxy-bbc57f5f6-wbpzm\" (UID: \"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.143600 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.143574    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d99c44c-128d-4fe8-8d3d-a8d655e18cd9-tls-certs\") pod \"kube-auth-proxy-bbc57f5f6-wbpzm\" (UID: \"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.143729 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.143635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgb6d\" (UniqueName: \"kubernetes.io/projected/9d99c44c-128d-4fe8-8d3d-a8d655e18cd9-kube-api-access-dgb6d\") pod \"kube-auth-proxy-bbc57f5f6-wbpzm\" (UID: \"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.143729 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.143669    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d99c44c-128d-4fe8-8d3d-a8d655e18cd9-tmp\") pod \"kube-auth-proxy-bbc57f5f6-wbpzm\" (UID: \"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.145891 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.145871    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d99c44c-128d-4fe8-8d3d-a8d655e18cd9-tmp\") pod \"kube-auth-proxy-bbc57f5f6-wbpzm\" (UID: \"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.146218 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.146198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d99c44c-128d-4fe8-8d3d-a8d655e18cd9-tls-certs\") pod \"kube-auth-proxy-bbc57f5f6-wbpzm\" (UID: \"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.160086 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.160057    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgb6d\" (UniqueName: \"kubernetes.io/projected/9d99c44c-128d-4fe8-8d3d-a8d655e18cd9-kube-api-access-dgb6d\") pod \"kube-auth-proxy-bbc57f5f6-wbpzm\" (UID: \"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9\") " pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.297428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.297366    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"
Jul 30 16:43:14.430583 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.430555    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm"]
Jul 30 16:43:14.432520 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:43:14.432495    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d99c44c_128d_4fe8_8d3d_a8d655e18cd9.slice/crio-9d1901527fbc011d877ef1cad91fac24507726735bb6065016e0d96bc0361b6b WatchSource:0}: Error finding container 9d1901527fbc011d877ef1cad91fac24507726735bb6065016e0d96bc0361b6b: Status 404 returned error can't find the container with id 9d1901527fbc011d877ef1cad91fac24507726735bb6065016e0d96bc0361b6b
Jul 30 16:43:14.696959 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:14.696929    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm" event={"ID":"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9","Type":"ContainerStarted","Data":"9d1901527fbc011d877ef1cad91fac24507726735bb6065016e0d96bc0361b6b"}
Jul 30 16:43:17.709979 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:17.709942    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm" event={"ID":"9d99c44c-128d-4fe8-8d3d-a8d655e18cd9","Type":"ContainerStarted","Data":"dad975b0148202afe12d6ec3bd1fa0e12591598bbf5560bc633336021e570e63"}
Jul 30 16:43:17.737967 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:43:17.737921    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-bbc57f5f6-wbpzm" podStartSLOduration=1.865959157 podStartE2EDuration="4.737907488s" totalImagesPullingTime="2.871948331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:43:17.73771379 +0000 UTC m=+425.340141415" watchObservedRunningTime="2026-07-30 16:43:17.737907488 +0000 UTC m=+425.340335113"
Jul 30 16:44:32.571268 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.571222    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-666c94bb4b-ttns4"]
Jul 30 16:44:32.573551 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.573530    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.578313 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.578289    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 16:44:32.578435 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.578415    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 16:44:32.579378 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.579286    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 16:44:32.579378 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.579300    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 16:44:32.579591 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.579384    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 16:44:32.583045 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.583024    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4sdqp\""
Jul 30 16:44:32.585802 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.585780    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 16:44:32.591308 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.591284    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-666c94bb4b-ttns4"]
Jul 30 16:44:32.609149 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.609126    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd427e00-b844-4128-86b0-bb12d0c26bd2-console-oauth-config\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.609233 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.609153    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-oauth-serving-cert\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.609233 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.609174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-service-ca\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.609309 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.609234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-trusted-ca-bundle\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.609309 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.609260    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxpxg\" (UniqueName: \"kubernetes.io/projected/dd427e00-b844-4128-86b0-bb12d0c26bd2-kube-api-access-gxpxg\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.609309 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.609296    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd427e00-b844-4128-86b0-bb12d0c26bd2-console-serving-cert\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.609400 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.609313    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-console-config\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.710085 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.710058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd427e00-b844-4128-86b0-bb12d0c26bd2-console-serving-cert\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.710198 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.710088    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-console-config\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.710198 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.710124    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd427e00-b844-4128-86b0-bb12d0c26bd2-console-oauth-config\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.710198 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.710140    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-oauth-serving-cert\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.710355 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.710255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-service-ca\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.710355 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.710312    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-trusted-ca-bundle\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.710448 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.710404    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxpxg\" (UniqueName: \"kubernetes.io/projected/dd427e00-b844-4128-86b0-bb12d0c26bd2-kube-api-access-gxpxg\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.711024 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.710994    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-console-config\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.711024 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.711009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-oauth-serving-cert\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.711200 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.711011    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-service-ca\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.711531 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.711507    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd427e00-b844-4128-86b0-bb12d0c26bd2-trusted-ca-bundle\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.712742 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.712711    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dd427e00-b844-4128-86b0-bb12d0c26bd2-console-oauth-config\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.712954 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.712933    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd427e00-b844-4128-86b0-bb12d0c26bd2-console-serving-cert\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.719393 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.719367    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxpxg\" (UniqueName: \"kubernetes.io/projected/dd427e00-b844-4128-86b0-bb12d0c26bd2-kube-api-access-gxpxg\") pod \"console-666c94bb4b-ttns4\" (UID: \"dd427e00-b844-4128-86b0-bb12d0c26bd2\") " pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:32.883213 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:32.883158    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:33.023824 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:33.023756    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-666c94bb4b-ttns4"]
Jul 30 16:44:33.025955 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:44:33.025925    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd427e00_b844_4128_86b0_bb12d0c26bd2.slice/crio-56a72325d140077933ed1042db2fd62c7d45c7eb7947eb45b6c13ba0a1b6e109 WatchSource:0}: Error finding container 56a72325d140077933ed1042db2fd62c7d45c7eb7947eb45b6c13ba0a1b6e109: Status 404 returned error can't find the container with id 56a72325d140077933ed1042db2fd62c7d45c7eb7947eb45b6c13ba0a1b6e109
Jul 30 16:44:34.025909 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:34.025875    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666c94bb4b-ttns4" event={"ID":"dd427e00-b844-4128-86b0-bb12d0c26bd2","Type":"ContainerStarted","Data":"6c1d2efc89bb151b717c50957fc2419f5e9711eba6e927f6140fd9aa04adeccf"}
Jul 30 16:44:34.026281 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:34.025922    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666c94bb4b-ttns4" event={"ID":"dd427e00-b844-4128-86b0-bb12d0c26bd2","Type":"ContainerStarted","Data":"56a72325d140077933ed1042db2fd62c7d45c7eb7947eb45b6c13ba0a1b6e109"}
Jul 30 16:44:34.047269 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:34.047110    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-666c94bb4b-ttns4" podStartSLOduration=2.047093403 podStartE2EDuration="2.047093403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:44:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:44:34.046720479 +0000 UTC m=+501.649148097" watchObservedRunningTime="2026-07-30 16:44:34.047093403 +0000 UTC m=+501.649521028"
Jul 30 16:44:42.883282 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:42.883241    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:42.883788 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:42.883313    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:42.888261 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:42.888231    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:44:43.064252 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:44:43.064229    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-666c94bb4b-ttns4"
Jul 30 16:45:03.060226 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.060197    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"]
Jul 30 16:45:03.062474 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.062455    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.067291 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.067271    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 16:45:03.067392 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.067308    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 30 16:45:03.067759 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.067743    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 30 16:45:03.068503 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.068487    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pvr8r\""
Jul 30 16:45:03.068596 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.068582    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 16:45:03.087990 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.087968    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"]
Jul 30 16:45:03.119953 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.119927    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c7a6f57-015e-4a1f-9a1e-04c4dfb73905-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mnzv2\" (UID: \"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.120054 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.119985    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxhn8\" (UniqueName: \"kubernetes.io/projected/7c7a6f57-015e-4a1f-9a1e-04c4dfb73905-kube-api-access-qxhn8\") pod \"kuadrant-console-plugin-66b5774985-mnzv2\" (UID: \"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.120122 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.120081    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c7a6f57-015e-4a1f-9a1e-04c4dfb73905-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mnzv2\" (UID: \"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.220767 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.220744    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c7a6f57-015e-4a1f-9a1e-04c4dfb73905-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mnzv2\" (UID: \"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.220857 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.220783    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxhn8\" (UniqueName: \"kubernetes.io/projected/7c7a6f57-015e-4a1f-9a1e-04c4dfb73905-kube-api-access-qxhn8\") pod \"kuadrant-console-plugin-66b5774985-mnzv2\" (UID: \"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.220857 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.220816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c7a6f57-015e-4a1f-9a1e-04c4dfb73905-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mnzv2\" (UID: \"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.221301 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.221282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7c7a6f57-015e-4a1f-9a1e-04c4dfb73905-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mnzv2\" (UID: \"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.223133 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.223109    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c7a6f57-015e-4a1f-9a1e-04c4dfb73905-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mnzv2\" (UID: \"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.230784 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.230765    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxhn8\" (UniqueName: \"kubernetes.io/projected/7c7a6f57-015e-4a1f-9a1e-04c4dfb73905-kube-api-access-qxhn8\") pod \"kuadrant-console-plugin-66b5774985-mnzv2\" (UID: \"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.372145 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.372095    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"
Jul 30 16:45:03.505374 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:03.505343    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2"]
Jul 30 16:45:03.507455 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:45:03.507417    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c7a6f57_015e_4a1f_9a1e_04c4dfb73905.slice/crio-bf905a1c77c28b28892ae5c0b46a3918dd911e25c51ac71952beb1b334183730 WatchSource:0}: Error finding container bf905a1c77c28b28892ae5c0b46a3918dd911e25c51ac71952beb1b334183730: Status 404 returned error can't find the container with id bf905a1c77c28b28892ae5c0b46a3918dd911e25c51ac71952beb1b334183730
Jul 30 16:45:04.146079 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:04.146044    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2" event={"ID":"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905","Type":"ContainerStarted","Data":"bf905a1c77c28b28892ae5c0b46a3918dd911e25c51ac71952beb1b334183730"}
Jul 30 16:45:25.247267 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:25.247230    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2" event={"ID":"7c7a6f57-015e-4a1f-9a1e-04c4dfb73905","Type":"ContainerStarted","Data":"ac7580ef4d59422695983c944d64e5891c826b153c57b09583d9015483cc14e6"}
Jul 30 16:45:25.263320 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:25.263269    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mnzv2" podStartSLOduration=1.069134554 podStartE2EDuration="22.263173719s" totalImagesPullingTime="21.194039165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:45:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:45:25.26256804 +0000 UTC m=+552.864995667" watchObservedRunningTime="2026-07-30 16:45:25.263173719 +0000 UTC m=+552.865601347"
Jul 30 16:45:55.218640 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.218593    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8brtz"]
Jul 30 16:45:55.323933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.323901    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8brtz"]
Jul 30 16:45:55.323933 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.323941    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8brtz"]
Jul 30 16:45:55.324127 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.323997    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:45:55.326632 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.326597    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 16:45:55.431151 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.431125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2edb7b6c-2b67-46ae-9930-01f8eaed2597-config-file\") pod \"limitador-limitador-64c94fd64f-8brtz\" (UID: \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:45:55.431260 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.431187    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lwj7\" (UniqueName: \"kubernetes.io/projected/2edb7b6c-2b67-46ae-9930-01f8eaed2597-kube-api-access-7lwj7\") pod \"limitador-limitador-64c94fd64f-8brtz\" (UID: \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:45:55.532035 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.531988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lwj7\" (UniqueName: \"kubernetes.io/projected/2edb7b6c-2b67-46ae-9930-01f8eaed2597-kube-api-access-7lwj7\") pod \"limitador-limitador-64c94fd64f-8brtz\" (UID: \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:45:55.532200 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.532034    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2edb7b6c-2b67-46ae-9930-01f8eaed2597-config-file\") pod \"limitador-limitador-64c94fd64f-8brtz\" (UID: \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:45:55.532561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.532542    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2edb7b6c-2b67-46ae-9930-01f8eaed2597-config-file\") pod \"limitador-limitador-64c94fd64f-8brtz\" (UID: \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:45:55.541186 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.541168    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lwj7\" (UniqueName: \"kubernetes.io/projected/2edb7b6c-2b67-46ae-9930-01f8eaed2597-kube-api-access-7lwj7\") pod \"limitador-limitador-64c94fd64f-8brtz\" (UID: \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:45:55.636114 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.636094    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:45:55.786925 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:55.786886    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8brtz"]
Jul 30 16:45:55.789560 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:45:55.789531    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2edb7b6c_2b67_46ae_9930_01f8eaed2597.slice/crio-95b02152fbb706a07ac6720cd4f21486e4b326e43012bbe9f734d3080dbf9925 WatchSource:0}: Error finding container 95b02152fbb706a07ac6720cd4f21486e4b326e43012bbe9f734d3080dbf9925: Status 404 returned error can't find the container with id 95b02152fbb706a07ac6720cd4f21486e4b326e43012bbe9f734d3080dbf9925
Jul 30 16:45:56.230904 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.230878    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-g8jfg"]
Jul 30 16:45:56.302755 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.302730    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g8jfg"]
Jul 30 16:45:56.302893 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.302827    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g8jfg"
Jul 30 16:45:56.310742 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.310724    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-p8bft\""
Jul 30 16:45:56.337518 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.337494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ptw7\" (UniqueName: \"kubernetes.io/projected/85e0e197-d7b4-4f50-b62b-b3d6141791e8-kube-api-access-2ptw7\") pod \"authorino-75d556bd77-g8jfg\" (UID: \"85e0e197-d7b4-4f50-b62b-b3d6141791e8\") " pod="kuadrant-system/authorino-75d556bd77-g8jfg"
Jul 30 16:45:56.378058 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.378029    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz" event={"ID":"2edb7b6c-2b67-46ae-9930-01f8eaed2597","Type":"ContainerStarted","Data":"95b02152fbb706a07ac6720cd4f21486e4b326e43012bbe9f734d3080dbf9925"}
Jul 30 16:45:56.438374 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.438350    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ptw7\" (UniqueName: \"kubernetes.io/projected/85e0e197-d7b4-4f50-b62b-b3d6141791e8-kube-api-access-2ptw7\") pod \"authorino-75d556bd77-g8jfg\" (UID: \"85e0e197-d7b4-4f50-b62b-b3d6141791e8\") " pod="kuadrant-system/authorino-75d556bd77-g8jfg"
Jul 30 16:45:56.446902 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.446884    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ptw7\" (UniqueName: \"kubernetes.io/projected/85e0e197-d7b4-4f50-b62b-b3d6141791e8-kube-api-access-2ptw7\") pod \"authorino-75d556bd77-g8jfg\" (UID: \"85e0e197-d7b4-4f50-b62b-b3d6141791e8\") " pod="kuadrant-system/authorino-75d556bd77-g8jfg"
Jul 30 16:45:56.611974 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.611823    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g8jfg"
Jul 30 16:45:56.776566 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:56.776540    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g8jfg"]
Jul 30 16:45:56.780369 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:45:56.780334    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85e0e197_d7b4_4f50_b62b_b3d6141791e8.slice/crio-0efbc8c3f069a356680344a491180121b1098b147bfdbf302ec11793464e651c WatchSource:0}: Error finding container 0efbc8c3f069a356680344a491180121b1098b147bfdbf302ec11793464e651c: Status 404 returned error can't find the container with id 0efbc8c3f069a356680344a491180121b1098b147bfdbf302ec11793464e651c
Jul 30 16:45:57.385851 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:45:57.385784    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g8jfg" event={"ID":"85e0e197-d7b4-4f50-b62b-b3d6141791e8","Type":"ContainerStarted","Data":"0efbc8c3f069a356680344a491180121b1098b147bfdbf302ec11793464e651c"}
Jul 30 16:46:00.406204 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:00.406164    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz" event={"ID":"2edb7b6c-2b67-46ae-9930-01f8eaed2597","Type":"ContainerStarted","Data":"28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900"}
Jul 30 16:46:00.406691 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:00.406419    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:46:00.408385 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:00.408353    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g8jfg" event={"ID":"85e0e197-d7b4-4f50-b62b-b3d6141791e8","Type":"ContainerStarted","Data":"f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d"}
Jul 30 16:46:00.428706 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:00.428668    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz" podStartSLOduration=0.928525724 podStartE2EDuration="5.428653264s" totalImagesPullingTime="4.50012754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:00.427037258 +0000 UTC m=+588.029464892" watchObservedRunningTime="2026-07-30 16:46:00.428653264 +0000 UTC m=+588.031080888"
Jul 30 16:46:00.442129 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:00.442092    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-g8jfg" podStartSLOduration=0.925270724 podStartE2EDuration="4.442080649s" totalImagesPullingTime="3.516809925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:00.4413248 +0000 UTC m=+588.043752429" watchObservedRunningTime="2026-07-30 16:46:00.442080649 +0000 UTC m=+588.044508276"
Jul 30 16:46:11.413481 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:11.413454    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:46:11.797856 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:11.797779    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8brtz"]
Jul 30 16:46:11.798015 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:11.797955    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz" podUID="2edb7b6c-2b67-46ae-9930-01f8eaed2597" containerName="limitador" containerID="cri-o://28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900" gracePeriod=30
Jul 30 16:46:12.341997 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.341975    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:46:12.461122 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.461098    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lwj7\" (UniqueName: \"kubernetes.io/projected/2edb7b6c-2b67-46ae-9930-01f8eaed2597-kube-api-access-7lwj7\") pod \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\" (UID: \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\") "
Jul 30 16:46:12.461507 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.461159    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2edb7b6c-2b67-46ae-9930-01f8eaed2597-config-file\") pod \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\" (UID: \"2edb7b6c-2b67-46ae-9930-01f8eaed2597\") "
Jul 30 16:46:12.461507 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.461480    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2edb7b6c-2b67-46ae-9930-01f8eaed2597-config-file" (OuterVolumeSpecName: "config-file") pod "2edb7b6c-2b67-46ae-9930-01f8eaed2597" (UID: "2edb7b6c-2b67-46ae-9930-01f8eaed2597"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 16:46:12.463151 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.463125    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2edb7b6c-2b67-46ae-9930-01f8eaed2597-kube-api-access-7lwj7" (OuterVolumeSpecName: "kube-api-access-7lwj7") pod "2edb7b6c-2b67-46ae-9930-01f8eaed2597" (UID: "2edb7b6c-2b67-46ae-9930-01f8eaed2597"). InnerVolumeSpecName "kube-api-access-7lwj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:46:12.468087 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.468062    2569 generic.go:356] "Generic (PLEG): container finished" podID="2edb7b6c-2b67-46ae-9930-01f8eaed2597" containerID="28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900" exitCode=0
Jul 30 16:46:12.468170 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.468108    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz" event={"ID":"2edb7b6c-2b67-46ae-9930-01f8eaed2597","Type":"ContainerDied","Data":"28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900"}
Jul 30 16:46:12.468170 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.468158    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz" event={"ID":"2edb7b6c-2b67-46ae-9930-01f8eaed2597","Type":"ContainerDied","Data":"95b02152fbb706a07ac6720cd4f21486e4b326e43012bbe9f734d3080dbf9925"}
Jul 30 16:46:12.468250 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.468165    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8brtz"
Jul 30 16:46:12.468250 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.468200    2569 scope.go:117] "RemoveContainer" containerID="28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900"
Jul 30 16:46:12.476410 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.476395    2569 scope.go:117] "RemoveContainer" containerID="28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900"
Jul 30 16:46:12.476702 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:46:12.476682    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900\": container with ID starting with 28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900 not found: ID does not exist" containerID="28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900"
Jul 30 16:46:12.476767 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.476711    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900"} err="failed to get container status \"28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900\": rpc error: code = NotFound desc = could not find container \"28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900\": container with ID starting with 28b51600c25582803d6a66864ad6dccd8c83bba584e0012b35c666138859f900 not found: ID does not exist"
Jul 30 16:46:12.488973 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.488949    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8brtz"]
Jul 30 16:46:12.493552 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.493526    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8brtz"]
Jul 30 16:46:12.562206 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.562183    2569 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2edb7b6c-2b67-46ae-9930-01f8eaed2597-config-file\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:46:12.562206 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.562203    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lwj7\" (UniqueName: \"kubernetes.io/projected/2edb7b6c-2b67-46ae-9930-01f8eaed2597-kube-api-access-7lwj7\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:46:12.999545 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:12.999506    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2edb7b6c-2b67-46ae-9930-01f8eaed2597" path="/var/lib/kubelet/pods/2edb7b6c-2b67-46ae-9930-01f8eaed2597/volumes"
Jul 30 16:46:15.712199 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.712162    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"]
Jul 30 16:46:15.712560 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.712481    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2edb7b6c-2b67-46ae-9930-01f8eaed2597" containerName="limitador"
Jul 30 16:46:15.712560 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.712497    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2edb7b6c-2b67-46ae-9930-01f8eaed2597" containerName="limitador"
Jul 30 16:46:15.712655 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.712564    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2edb7b6c-2b67-46ae-9930-01f8eaed2597" containerName="limitador"
Jul 30 16:46:15.714874 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.714838    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:15.717688 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.717665    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 16:46:15.717803 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.717712    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 30 16:46:15.718460 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.718441    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fr79m\""
Jul 30 16:46:15.718515 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.718441    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 16:46:15.728268 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.728246    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"]
Jul 30 16:46:15.784662 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.784639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/408fbd2d-5ac5-46c8-90c4-ca336153dbb8-data\") pod \"postgres-69fb4594cf-bfmmb\" (UID: \"408fbd2d-5ac5-46c8-90c4-ca336153dbb8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:15.784752 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.784690    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szrz2\" (UniqueName: \"kubernetes.io/projected/408fbd2d-5ac5-46c8-90c4-ca336153dbb8-kube-api-access-szrz2\") pod \"postgres-69fb4594cf-bfmmb\" (UID: \"408fbd2d-5ac5-46c8-90c4-ca336153dbb8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:15.885525 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.885504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szrz2\" (UniqueName: \"kubernetes.io/projected/408fbd2d-5ac5-46c8-90c4-ca336153dbb8-kube-api-access-szrz2\") pod \"postgres-69fb4594cf-bfmmb\" (UID: \"408fbd2d-5ac5-46c8-90c4-ca336153dbb8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:15.885633 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.885551    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/408fbd2d-5ac5-46c8-90c4-ca336153dbb8-data\") pod \"postgres-69fb4594cf-bfmmb\" (UID: \"408fbd2d-5ac5-46c8-90c4-ca336153dbb8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:15.885836 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.885821    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/408fbd2d-5ac5-46c8-90c4-ca336153dbb8-data\") pod \"postgres-69fb4594cf-bfmmb\" (UID: \"408fbd2d-5ac5-46c8-90c4-ca336153dbb8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:15.895295 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:15.895271    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szrz2\" (UniqueName: \"kubernetes.io/projected/408fbd2d-5ac5-46c8-90c4-ca336153dbb8-kube-api-access-szrz2\") pod \"postgres-69fb4594cf-bfmmb\" (UID: \"408fbd2d-5ac5-46c8-90c4-ca336153dbb8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:16.025003 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:16.024956    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:16.151179 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:16.151156    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"]
Jul 30 16:46:16.153459 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:46:16.153428    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408fbd2d_5ac5_46c8_90c4_ca336153dbb8.slice/crio-b009ee13bd02584963b45518126c835a7f0c2100f9b2494901c501bc3bf80347 WatchSource:0}: Error finding container b009ee13bd02584963b45518126c835a7f0c2100f9b2494901c501bc3bf80347: Status 404 returned error can't find the container with id b009ee13bd02584963b45518126c835a7f0c2100f9b2494901c501bc3bf80347
Jul 30 16:46:16.486090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:16.486053    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb" event={"ID":"408fbd2d-5ac5-46c8-90c4-ca336153dbb8","Type":"ContainerStarted","Data":"b009ee13bd02584963b45518126c835a7f0c2100f9b2494901c501bc3bf80347"}
Jul 30 16:46:21.517117 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:21.517076    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb" event={"ID":"408fbd2d-5ac5-46c8-90c4-ca336153dbb8","Type":"ContainerStarted","Data":"dd5a03ce94d480e709c89e1f88c19bd440c03628a51ddf28c7d5c3de3b0d4e48"}
Jul 30 16:46:21.517474 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:21.517191    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:21.534257 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:21.534216    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb" podStartSLOduration=1.897660285 podStartE2EDuration="6.534203337s" totalImagesPullingTime="4.636543052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:46:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:21.533763512 +0000 UTC m=+609.136191138" watchObservedRunningTime="2026-07-30 16:46:21.534203337 +0000 UTC m=+609.136630963"
Jul 30 16:46:27.548538 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:27.548508    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bfmmb"
Jul 30 16:46:29.926831 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:29.926792    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g8jfg"]
Jul 30 16:46:29.927287 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:29.926984    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-g8jfg" podUID="85e0e197-d7b4-4f50-b62b-b3d6141791e8" containerName="authorino" containerID="cri-o://f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d" gracePeriod=30
Jul 30 16:46:30.175407 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.175385    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g8jfg"
Jul 30 16:46:30.298850 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.298792    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ptw7\" (UniqueName: \"kubernetes.io/projected/85e0e197-d7b4-4f50-b62b-b3d6141791e8-kube-api-access-2ptw7\") pod \"85e0e197-d7b4-4f50-b62b-b3d6141791e8\" (UID: \"85e0e197-d7b4-4f50-b62b-b3d6141791e8\") "
Jul 30 16:46:30.300686 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.300663    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85e0e197-d7b4-4f50-b62b-b3d6141791e8-kube-api-access-2ptw7" (OuterVolumeSpecName: "kube-api-access-2ptw7") pod "85e0e197-d7b4-4f50-b62b-b3d6141791e8" (UID: "85e0e197-d7b4-4f50-b62b-b3d6141791e8"). InnerVolumeSpecName "kube-api-access-2ptw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:46:30.399796 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.399766    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ptw7\" (UniqueName: \"kubernetes.io/projected/85e0e197-d7b4-4f50-b62b-b3d6141791e8-kube-api-access-2ptw7\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:46:30.558160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.558086    2569 generic.go:356] "Generic (PLEG): container finished" podID="85e0e197-d7b4-4f50-b62b-b3d6141791e8" containerID="f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d" exitCode=0
Jul 30 16:46:30.558160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.558145    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g8jfg"
Jul 30 16:46:30.558307 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.558184    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g8jfg" event={"ID":"85e0e197-d7b4-4f50-b62b-b3d6141791e8","Type":"ContainerDied","Data":"f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d"}
Jul 30 16:46:30.558307 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.558243    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g8jfg" event={"ID":"85e0e197-d7b4-4f50-b62b-b3d6141791e8","Type":"ContainerDied","Data":"0efbc8c3f069a356680344a491180121b1098b147bfdbf302ec11793464e651c"}
Jul 30 16:46:30.558307 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.558265    2569 scope.go:117] "RemoveContainer" containerID="f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d"
Jul 30 16:46:30.567064 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.567047    2569 scope.go:117] "RemoveContainer" containerID="f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d"
Jul 30 16:46:30.567357 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:46:30.567334    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d\": container with ID starting with f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d not found: ID does not exist" containerID="f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d"
Jul 30 16:46:30.567408 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.567365    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d"} err="failed to get container status \"f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d\": rpc error: code = NotFound desc = could not find container \"f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d\": container with ID starting with f39d79ff2b6efcfc3d0d0e937cd83e8abe80f0ba7ec6a6a3b781330c8a01457d not found: ID does not exist"
Jul 30 16:46:30.589211 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.589185    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g8jfg"]
Jul 30 16:46:30.597068 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.597042    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g8jfg"]
Jul 30 16:46:30.999808 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:30.999779    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85e0e197-d7b4-4f50-b62b-b3d6141791e8" path="/var/lib/kubelet/pods/85e0e197-d7b4-4f50-b62b-b3d6141791e8/volumes"
Jul 30 16:46:34.609000 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.608924    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fd596b994-wssm5"]
Jul 30 16:46:34.609425 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.609404    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85e0e197-d7b4-4f50-b62b-b3d6141791e8" containerName="authorino"
Jul 30 16:46:34.609494 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.609428    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="85e0e197-d7b4-4f50-b62b-b3d6141791e8" containerName="authorino"
Jul 30 16:46:34.609542 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.609503    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="85e0e197-d7b4-4f50-b62b-b3d6141791e8" containerName="authorino"
Jul 30 16:46:34.708915 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.708890    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fd596b994-wssm5"]
Jul 30 16:46:34.709079 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.708995    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:34.711697 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.711673    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 16:46:34.711806 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.711701    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 16:46:34.711849 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.711801    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mww4n\""
Jul 30 16:46:34.804336 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.804311    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f6d974979-hljgq"]
Jul 30 16:46:34.820794 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.820777    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f6d974979-hljgq"]
Jul 30 16:46:34.820893 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.820859    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:34.834101 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.834076    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e96f4262-c4c1-4b8a-9165-3e76722f3438-webhook-cert\") pod \"maas-controller-6fd596b994-wssm5\" (UID: \"e96f4262-c4c1-4b8a-9165-3e76722f3438\") " pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:34.834180 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.834124    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94grg\" (UniqueName: \"kubernetes.io/projected/e96f4262-c4c1-4b8a-9165-3e76722f3438-kube-api-access-94grg\") pod \"maas-controller-6fd596b994-wssm5\" (UID: \"e96f4262-c4c1-4b8a-9165-3e76722f3438\") " pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:34.935196 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.935168    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31a102a4-f550-4add-b3d2-4945c93f8695-webhook-cert\") pod \"maas-controller-5f6d974979-hljgq\" (UID: \"31a102a4-f550-4add-b3d2-4945c93f8695\") " pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:34.935336 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.935199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kjhm\" (UniqueName: \"kubernetes.io/projected/31a102a4-f550-4add-b3d2-4945c93f8695-kube-api-access-9kjhm\") pod \"maas-controller-5f6d974979-hljgq\" (UID: \"31a102a4-f550-4add-b3d2-4945c93f8695\") " pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:34.935336 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.935284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e96f4262-c4c1-4b8a-9165-3e76722f3438-webhook-cert\") pod \"maas-controller-6fd596b994-wssm5\" (UID: \"e96f4262-c4c1-4b8a-9165-3e76722f3438\") " pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:34.935336 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.935320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94grg\" (UniqueName: \"kubernetes.io/projected/e96f4262-c4c1-4b8a-9165-3e76722f3438-kube-api-access-94grg\") pod \"maas-controller-6fd596b994-wssm5\" (UID: \"e96f4262-c4c1-4b8a-9165-3e76722f3438\") " pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:34.937753 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.937727    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e96f4262-c4c1-4b8a-9165-3e76722f3438-webhook-cert\") pod \"maas-controller-6fd596b994-wssm5\" (UID: \"e96f4262-c4c1-4b8a-9165-3e76722f3438\") " pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:34.944570 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:34.944552    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94grg\" (UniqueName: \"kubernetes.io/projected/e96f4262-c4c1-4b8a-9165-3e76722f3438-kube-api-access-94grg\") pod \"maas-controller-6fd596b994-wssm5\" (UID: \"e96f4262-c4c1-4b8a-9165-3e76722f3438\") " pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:35.018150 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.018132    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:35.036445 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.036418    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31a102a4-f550-4add-b3d2-4945c93f8695-webhook-cert\") pod \"maas-controller-5f6d974979-hljgq\" (UID: \"31a102a4-f550-4add-b3d2-4945c93f8695\") " pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:35.036529 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.036450    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kjhm\" (UniqueName: \"kubernetes.io/projected/31a102a4-f550-4add-b3d2-4945c93f8695-kube-api-access-9kjhm\") pod \"maas-controller-5f6d974979-hljgq\" (UID: \"31a102a4-f550-4add-b3d2-4945c93f8695\") " pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:35.038464 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.038442    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31a102a4-f550-4add-b3d2-4945c93f8695-webhook-cert\") pod \"maas-controller-5f6d974979-hljgq\" (UID: \"31a102a4-f550-4add-b3d2-4945c93f8695\") " pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:35.044910 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.044723    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kjhm\" (UniqueName: \"kubernetes.io/projected/31a102a4-f550-4add-b3d2-4945c93f8695-kube-api-access-9kjhm\") pod \"maas-controller-5f6d974979-hljgq\" (UID: \"31a102a4-f550-4add-b3d2-4945c93f8695\") " pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:35.130983 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.130958    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:35.166950 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.166921    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fd596b994-wssm5"]
Jul 30 16:46:35.169373 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:46:35.169348    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96f4262_c4c1_4b8a_9165_3e76722f3438.slice/crio-0608c5bb744a132e09dd87ee1ebb3a7356a3f531dd0874c1f0d9784ac5507b8d WatchSource:0}: Error finding container 0608c5bb744a132e09dd87ee1ebb3a7356a3f531dd0874c1f0d9784ac5507b8d: Status 404 returned error can't find the container with id 0608c5bb744a132e09dd87ee1ebb3a7356a3f531dd0874c1f0d9784ac5507b8d
Jul 30 16:46:35.282398 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.282340    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f6d974979-hljgq"]
Jul 30 16:46:35.284415 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:46:35.284393    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a102a4_f550_4add_b3d2_4945c93f8695.slice/crio-a711a66019674e7ceaa373a2f4c586d13756b9d6381c5b1bff04eb8e483a41d3 WatchSource:0}: Error finding container a711a66019674e7ceaa373a2f4c586d13756b9d6381c5b1bff04eb8e483a41d3: Status 404 returned error can't find the container with id a711a66019674e7ceaa373a2f4c586d13756b9d6381c5b1bff04eb8e483a41d3
Jul 30 16:46:35.584976 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.584913    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-hljgq" event={"ID":"31a102a4-f550-4add-b3d2-4945c93f8695","Type":"ContainerStarted","Data":"a711a66019674e7ceaa373a2f4c586d13756b9d6381c5b1bff04eb8e483a41d3"}
Jul 30 16:46:35.586006 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:35.585972    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd596b994-wssm5" event={"ID":"e96f4262-c4c1-4b8a-9165-3e76722f3438","Type":"ContainerStarted","Data":"0608c5bb744a132e09dd87ee1ebb3a7356a3f531dd0874c1f0d9784ac5507b8d"}
Jul 30 16:46:38.602381 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:38.602344    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd596b994-wssm5" event={"ID":"e96f4262-c4c1-4b8a-9165-3e76722f3438","Type":"ContainerStarted","Data":"20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de"}
Jul 30 16:46:38.602945 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:38.602405    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:38.603834 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:38.603805    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-hljgq" event={"ID":"31a102a4-f550-4add-b3d2-4945c93f8695","Type":"ContainerStarted","Data":"694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac"}
Jul 30 16:46:38.603951 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:38.603934    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:38.655125 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:38.655078    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fd596b994-wssm5" podStartSLOduration=1.663871825 podStartE2EDuration="4.655064597s" totalImagesPullingTime="2.991192772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:38.653064611 +0000 UTC m=+626.255492237" watchObservedRunningTime="2026-07-30 16:46:38.655064597 +0000 UTC m=+626.257492223"
Jul 30 16:46:38.685138 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:38.685097    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f6d974979-hljgq" podStartSLOduration=1.8158138639999999 podStartE2EDuration="4.685084758s" totalImagesPullingTime="2.869270894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:46:38.682955549 +0000 UTC m=+626.285383173" watchObservedRunningTime="2026-07-30 16:46:38.685084758 +0000 UTC m=+626.287512383"
Jul 30 16:46:49.609978 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:49.609946    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:46:49.610429 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:49.610030    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:49.684923 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:49.684893    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fd596b994-wssm5"]
Jul 30 16:46:49.685084 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:49.685068    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6fd596b994-wssm5" podUID="e96f4262-c4c1-4b8a-9165-3e76722f3438" containerName="manager" containerID="cri-o://20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de" gracePeriod=30
Jul 30 16:46:49.919876 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:49.919853    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:50.053560 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.053533    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e96f4262-c4c1-4b8a-9165-3e76722f3438-webhook-cert\") pod \"e96f4262-c4c1-4b8a-9165-3e76722f3438\" (UID: \"e96f4262-c4c1-4b8a-9165-3e76722f3438\") "
Jul 30 16:46:50.053703 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.053566    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94grg\" (UniqueName: \"kubernetes.io/projected/e96f4262-c4c1-4b8a-9165-3e76722f3438-kube-api-access-94grg\") pod \"e96f4262-c4c1-4b8a-9165-3e76722f3438\" (UID: \"e96f4262-c4c1-4b8a-9165-3e76722f3438\") "
Jul 30 16:46:50.055572 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.055546    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e96f4262-c4c1-4b8a-9165-3e76722f3438-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e96f4262-c4c1-4b8a-9165-3e76722f3438" (UID: "e96f4262-c4c1-4b8a-9165-3e76722f3438"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:46:50.055661 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.055638    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e96f4262-c4c1-4b8a-9165-3e76722f3438-kube-api-access-94grg" (OuterVolumeSpecName: "kube-api-access-94grg") pod "e96f4262-c4c1-4b8a-9165-3e76722f3438" (UID: "e96f4262-c4c1-4b8a-9165-3e76722f3438"). InnerVolumeSpecName "kube-api-access-94grg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:46:50.155122 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.155074    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e96f4262-c4c1-4b8a-9165-3e76722f3438-webhook-cert\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:46:50.155122 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.155097    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94grg\" (UniqueName: \"kubernetes.io/projected/e96f4262-c4c1-4b8a-9165-3e76722f3438-kube-api-access-94grg\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:46:50.663378 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.663349    2569 generic.go:356] "Generic (PLEG): container finished" podID="e96f4262-c4c1-4b8a-9165-3e76722f3438" containerID="20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de" exitCode=0
Jul 30 16:46:50.663779 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.663395    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd596b994-wssm5"
Jul 30 16:46:50.663779 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.663403    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd596b994-wssm5" event={"ID":"e96f4262-c4c1-4b8a-9165-3e76722f3438","Type":"ContainerDied","Data":"20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de"}
Jul 30 16:46:50.663779 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.663445    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd596b994-wssm5" event={"ID":"e96f4262-c4c1-4b8a-9165-3e76722f3438","Type":"ContainerDied","Data":"0608c5bb744a132e09dd87ee1ebb3a7356a3f531dd0874c1f0d9784ac5507b8d"}
Jul 30 16:46:50.663779 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.663467    2569 scope.go:117] "RemoveContainer" containerID="20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de"
Jul 30 16:46:50.672386 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.672368    2569 scope.go:117] "RemoveContainer" containerID="20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de"
Jul 30 16:46:50.672677 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:46:50.672657    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de\": container with ID starting with 20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de not found: ID does not exist" containerID="20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de"
Jul 30 16:46:50.672743 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.672686    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de"} err="failed to get container status \"20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de\": rpc error: code = NotFound desc = could not find container \"20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de\": container with ID starting with 20aa4043c2bde4c737331d6dde518417b852e7ab06253e6d74b0bad4906497de not found: ID does not exist"
Jul 30 16:46:50.684219 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.684194    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fd596b994-wssm5"]
Jul 30 16:46:50.690860 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:50.690839    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6fd596b994-wssm5"]
Jul 30 16:46:51.000759 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:46:51.000705    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e96f4262-c4c1-4b8a-9165-3e76722f3438" path="/var/lib/kubelet/pods/e96f4262-c4c1-4b8a-9165-3e76722f3438/volumes"
Jul 30 16:47:11.778141 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.778105    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"]
Jul 30 16:47:11.778574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.778403    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e96f4262-c4c1-4b8a-9165-3e76722f3438" containerName="manager"
Jul 30 16:47:11.778574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.778415    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e96f4262-c4c1-4b8a-9165-3e76722f3438" containerName="manager"
Jul 30 16:47:11.778574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.778477    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e96f4262-c4c1-4b8a-9165-3e76722f3438" containerName="manager"
Jul 30 16:47:11.786566 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.786549    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:11.789113 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.789077    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9cvkk\""
Jul 30 16:47:11.789221 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.789139    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 30 16:47:11.793513 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.793492    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"]
Jul 30 16:47:11.905374 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.905353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b36c00ab-ef8b-4328-a963-624cc8b1c38a-maas-api-tls\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:11.905492 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.905385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8x2q\" (UniqueName: \"kubernetes.io/projected/b36c00ab-ef8b-4328-a963-624cc8b1c38a-kube-api-access-d8x2q\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:11.905492 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:11.905408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b36c00ab-ef8b-4328-a963-624cc8b1c38a-openshift-service-ca\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:12.006527 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.006506    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b36c00ab-ef8b-4328-a963-624cc8b1c38a-openshift-service-ca\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:12.006645 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.006568    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b36c00ab-ef8b-4328-a963-624cc8b1c38a-maas-api-tls\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:12.006645 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.006591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8x2q\" (UniqueName: \"kubernetes.io/projected/b36c00ab-ef8b-4328-a963-624cc8b1c38a-kube-api-access-d8x2q\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:12.006762 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:47:12.006721    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 30 16:47:12.006840 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:47:12.006828    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b36c00ab-ef8b-4328-a963-624cc8b1c38a-maas-api-tls podName:b36c00ab-ef8b-4328-a963-624cc8b1c38a nodeName:}" failed. No retries permitted until 2026-07-30 16:47:12.506806512 +0000 UTC m=+660.109234123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b36c00ab-ef8b-4328-a963-624cc8b1c38a-maas-api-tls") pod "maas-api-ff4449fc7-mm6cq" (UID: "b36c00ab-ef8b-4328-a963-624cc8b1c38a") : secret "maas-api-serving-cert" not found
Jul 30 16:47:12.007076 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.007056    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b36c00ab-ef8b-4328-a963-624cc8b1c38a-openshift-service-ca\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:12.015335 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.015318    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8x2q\" (UniqueName: \"kubernetes.io/projected/b36c00ab-ef8b-4328-a963-624cc8b1c38a-kube-api-access-d8x2q\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:12.465845 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.465814    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"]
Jul 30 16:47:12.469622 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.469593    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:12.473432 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.473412    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-r6q4n\""
Jul 30 16:47:12.473549 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.473411    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 30 16:47:12.483844 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.483816    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"]
Jul 30 16:47:12.511971 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.511951    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b36c00ab-ef8b-4328-a963-624cc8b1c38a-maas-api-tls\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:12.512092 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.512020    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kstr\" (UniqueName: \"kubernetes.io/projected/9093bfce-5746-4229-817f-3ceabe0b6b5e-kube-api-access-2kstr\") pod \"payload-pre-processing-687dbb6cf6-m87vz\" (UID: \"9093bfce-5746-4229-817f-3ceabe0b6b5e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:12.512092 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.512047    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9093bfce-5746-4229-817f-3ceabe0b6b5e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-m87vz\" (UID: \"9093bfce-5746-4229-817f-3ceabe0b6b5e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:12.514172 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.514152    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b36c00ab-ef8b-4328-a963-624cc8b1c38a-maas-api-tls\") pod \"maas-api-ff4449fc7-mm6cq\" (UID: \"b36c00ab-ef8b-4328-a963-624cc8b1c38a\") " pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:12.596077 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.596052    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wgxkq"]
Jul 30 16:47:12.599600 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.599587    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:12.612426 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.612399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kstr\" (UniqueName: \"kubernetes.io/projected/9093bfce-5746-4229-817f-3ceabe0b6b5e-kube-api-access-2kstr\") pod \"payload-pre-processing-687dbb6cf6-m87vz\" (UID: \"9093bfce-5746-4229-817f-3ceabe0b6b5e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:12.612535 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.612436    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9093bfce-5746-4229-817f-3ceabe0b6b5e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-m87vz\" (UID: \"9093bfce-5746-4229-817f-3ceabe0b6b5e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:12.613272 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.613248    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9093bfce-5746-4229-817f-3ceabe0b6b5e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-m87vz\" (UID: \"9093bfce-5746-4229-817f-3ceabe0b6b5e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:12.613939 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.613918    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wgxkq"]
Jul 30 16:47:12.625073 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.625054    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kstr\" (UniqueName: \"kubernetes.io/projected/9093bfce-5746-4229-817f-3ceabe0b6b5e-kube-api-access-2kstr\") pod \"payload-pre-processing-687dbb6cf6-m87vz\" (UID: \"9093bfce-5746-4229-817f-3ceabe0b6b5e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:12.697485 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.697455    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:12.713241 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.713218    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d438334-8a53-4796-8080-8ce8dabe3afc-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wgxkq\" (UID: \"1d438334-8a53-4796-8080-8ce8dabe3afc\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:12.713349 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.713282    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx5rp\" (UniqueName: \"kubernetes.io/projected/1d438334-8a53-4796-8080-8ce8dabe3afc-kube-api-access-kx5rp\") pod \"payload-processing-5cc6cd444b-wgxkq\" (UID: \"1d438334-8a53-4796-8080-8ce8dabe3afc\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:12.782062 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.782038    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-r6q4n\""
Jul 30 16:47:12.789851 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.789823    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:12.820420 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.820385    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d438334-8a53-4796-8080-8ce8dabe3afc-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wgxkq\" (UID: \"1d438334-8a53-4796-8080-8ce8dabe3afc\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:12.820659 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.820488    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx5rp\" (UniqueName: \"kubernetes.io/projected/1d438334-8a53-4796-8080-8ce8dabe3afc-kube-api-access-kx5rp\") pod \"payload-processing-5cc6cd444b-wgxkq\" (UID: \"1d438334-8a53-4796-8080-8ce8dabe3afc\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:12.821275 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.821252    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d438334-8a53-4796-8080-8ce8dabe3afc-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-wgxkq\" (UID: \"1d438334-8a53-4796-8080-8ce8dabe3afc\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:12.829817 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.829797    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx5rp\" (UniqueName: \"kubernetes.io/projected/1d438334-8a53-4796-8080-8ce8dabe3afc-kube-api-access-kx5rp\") pod \"payload-processing-5cc6cd444b-wgxkq\" (UID: \"1d438334-8a53-4796-8080-8ce8dabe3afc\") " pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:12.834308 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.834285    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"]
Jul 30 16:47:12.838298 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:47:12.838262    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36c00ab_ef8b_4328_a963_624cc8b1c38a.slice/crio-705082aa751849fc0f43cddd097e48b1cada3df1ec5f14b1f58c787deb9c6091 WatchSource:0}: Error finding container 705082aa751849fc0f43cddd097e48b1cada3df1ec5f14b1f58c787deb9c6091: Status 404 returned error can't find the container with id 705082aa751849fc0f43cddd097e48b1cada3df1ec5f14b1f58c787deb9c6091
Jul 30 16:47:12.909100 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.909072    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:12.935363 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:47:12.935330    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9093bfce_5746_4229_817f_3ceabe0b6b5e.slice/crio-109d70f114dd60d3a82e9eee5d6234cd7762db5e584de9553bce98fe8130ec5a WatchSource:0}: Error finding container 109d70f114dd60d3a82e9eee5d6234cd7762db5e584de9553bce98fe8130ec5a: Status 404 returned error can't find the container with id 109d70f114dd60d3a82e9eee5d6234cd7762db5e584de9553bce98fe8130ec5a
Jul 30 16:47:12.946378 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:12.946350    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"]
Jul 30 16:47:13.115985 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:13.115941    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-wgxkq"]
Jul 30 16:47:13.117878 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:47:13.117853    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d438334_8a53_4796_8080_8ce8dabe3afc.slice/crio-453f77634cb8e2dc7f14183aa851402696e32138110a0d78bae93128cfa35c16 WatchSource:0}: Error finding container 453f77634cb8e2dc7f14183aa851402696e32138110a0d78bae93128cfa35c16: Status 404 returned error can't find the container with id 453f77634cb8e2dc7f14183aa851402696e32138110a0d78bae93128cfa35c16
Jul 30 16:47:13.780559 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:13.776513    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq" event={"ID":"1d438334-8a53-4796-8080-8ce8dabe3afc","Type":"ContainerStarted","Data":"453f77634cb8e2dc7f14183aa851402696e32138110a0d78bae93128cfa35c16"}
Jul 30 16:47:13.780559 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:13.778214    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz" event={"ID":"9093bfce-5746-4229-817f-3ceabe0b6b5e","Type":"ContainerStarted","Data":"109d70f114dd60d3a82e9eee5d6234cd7762db5e584de9553bce98fe8130ec5a"}
Jul 30 16:47:13.780559 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:13.780481    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq" event={"ID":"b36c00ab-ef8b-4328-a963-624cc8b1c38a","Type":"ContainerStarted","Data":"705082aa751849fc0f43cddd097e48b1cada3df1ec5f14b1f58c787deb9c6091"}
Jul 30 16:47:15.791592 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:15.791561    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq" event={"ID":"1d438334-8a53-4796-8080-8ce8dabe3afc","Type":"ContainerStarted","Data":"92d62a1b644d843355db69de5f6f213dc49564852a16cbf53c7bd18da4fa5799"}
Jul 30 16:47:15.792015 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:15.791635    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:15.793093 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:15.793063    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz" event={"ID":"9093bfce-5746-4229-817f-3ceabe0b6b5e","Type":"ContainerStarted","Data":"12c018349a736ab6e3decc7c17583fe9ed02d96fa73272b55cbde3fe3e56d61c"}
Jul 30 16:47:15.793380 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:15.793118    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:15.794667 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:15.794644    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq" event={"ID":"b36c00ab-ef8b-4328-a963-624cc8b1c38a","Type":"ContainerStarted","Data":"e3f7bedf92407e19bfa2e032b702c9ac493159bbe927141137e6dd6dfd7e64f7"}
Jul 30 16:47:15.794773 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:15.794711    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:15.808543 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:15.808499    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq" podStartSLOduration=1.274766777 podStartE2EDuration="3.80848559s" totalImagesPullingTime="2.533718813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:47:15.807455222 +0000 UTC m=+663.409882849" watchObservedRunningTime="2026-07-30 16:47:15.80848559 +0000 UTC m=+663.410913194"
Jul 30 16:47:15.824705 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:15.824667    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz" podStartSLOduration=1.112652704 podStartE2EDuration="3.824656005s" totalImagesPullingTime="2.712003301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:47:15.824034445 +0000 UTC m=+663.426462075" watchObservedRunningTime="2026-07-30 16:47:15.824656005 +0000 UTC m=+663.427083631"
Jul 30 16:47:15.841564 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:15.841529    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq" podStartSLOduration=2.087114482 podStartE2EDuration="4.84151719s" totalImagesPullingTime="2.754402708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:47:15.840020902 +0000 UTC m=+663.442448529" watchObservedRunningTime="2026-07-30 16:47:15.84151719 +0000 UTC m=+663.443944817"
Jul 30 16:47:21.804835 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:21.804794    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-ff4449fc7-mm6cq"
Jul 30 16:47:26.801792 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:26.801750    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-m87vz"
Jul 30 16:47:26.802285 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:26.801833    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-wgxkq"
Jul 30 16:47:53.839516 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.839482    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"]
Jul 30 16:47:53.844847 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.844829    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.848431 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.848409    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sqbq4\""
Jul 30 16:47:53.848595 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.848504    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 16:47:53.848595 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.848505    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 16:47:53.848595 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.848505    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 30 16:47:53.851681 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.851659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.851788 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.851695    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kz6d\" (UniqueName: \"kubernetes.io/projected/c3febbc4-7bac-461a-b52f-c4edb22e2b36-kube-api-access-9kz6d\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.851837 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.851792    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.851890 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.851838    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.851943 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.851914    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.851994 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.851973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3febbc4-7bac-461a-b52f-c4edb22e2b36-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.859127 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.859104    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"]
Jul 30 16:47:53.953093 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.953069    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.953196 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.953101    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3febbc4-7bac-461a-b52f-c4edb22e2b36-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.953196 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.953134    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.953196 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.953158    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kz6d\" (UniqueName: \"kubernetes.io/projected/c3febbc4-7bac-461a-b52f-c4edb22e2b36-kube-api-access-9kz6d\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.953354 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.953196    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.953354 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.953218    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.953560 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.953541    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.953640 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.953586    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.953640 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.953601    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.955198 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.955176    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3febbc4-7bac-461a-b52f-c4edb22e2b36-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.955476 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.955460    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3febbc4-7bac-461a-b52f-c4edb22e2b36-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:53.962546 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:53.962523    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kz6d\" (UniqueName: \"kubernetes.io/projected/c3febbc4-7bac-461a-b52f-c4edb22e2b36-kube-api-access-9kz6d\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8slw6\" (UID: \"c3febbc4-7bac-461a-b52f-c4edb22e2b36\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:54.154889 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:54.154864    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:47:54.289494 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:54.289362    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"]
Jul 30 16:47:54.292113 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:47:54.292086    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3febbc4_7bac_461a_b52f_c4edb22e2b36.slice/crio-225d9c143a9fed16d5fe184ef8e2f77b28661c5df18d4fe895ecbbe48b748920 WatchSource:0}: Error finding container 225d9c143a9fed16d5fe184ef8e2f77b28661c5df18d4fe895ecbbe48b748920: Status 404 returned error can't find the container with id 225d9c143a9fed16d5fe184ef8e2f77b28661c5df18d4fe895ecbbe48b748920
Jul 30 16:47:54.293375 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:54.293356    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 16:47:54.972595 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:54.972546    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6" event={"ID":"c3febbc4-7bac-461a-b52f-c4edb22e2b36","Type":"ContainerStarted","Data":"225d9c143a9fed16d5fe184ef8e2f77b28661c5df18d4fe895ecbbe48b748920"}
Jul 30 16:47:55.978397 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:55.978355    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6" event={"ID":"c3febbc4-7bac-461a-b52f-c4edb22e2b36","Type":"ContainerStarted","Data":"7e502a0366185eafbbea059ed56d9ae5362c6c0ea6b279b267bdb9965a690c7d"}
Jul 30 16:47:55.997874 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:47:55.997825    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6" podStartSLOduration=2.00077059 podStartE2EDuration="2.997808837s" totalImagesPullingTime="997.038247ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:47:55.995505782 +0000 UTC m=+703.597933408" watchObservedRunningTime="2026-07-30 16:47:55.997808837 +0000 UTC m=+703.600236521"
Jul 30 16:48:04.155677 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:04.155638    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:48:04.156063 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:04.155719    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:48:04.167937 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:04.167915    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:48:05.029749 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:05.029722    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8slw6"
Jul 30 16:48:58.912441 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.912402    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"]
Jul 30 16:48:58.916041 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.916017    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:58.918597 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.918574    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 30 16:48:58.927438 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.927411    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"]
Jul 30 16:48:58.964356 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.964331    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:58.964458 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.964360    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:58.964458 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.964381    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:58.964458 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.964429    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vtn7\" (UniqueName: \"kubernetes.io/projected/a853848e-1449-4ae2-834b-912322c73848-kube-api-access-2vtn7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:58.964574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.964502    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:58.964574 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:58.964521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a853848e-1449-4ae2-834b-912322c73848-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.064865 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.064833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.064865 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.064864    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a853848e-1449-4ae2-834b-912322c73848-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.065025 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.064894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.065025 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.064910    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.065025 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.064934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.065025 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.064974    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtn7\" (UniqueName: \"kubernetes.io/projected/a853848e-1449-4ae2-834b-912322c73848-kube-api-access-2vtn7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.065208 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.065186    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.065315 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.065295    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.065388 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.065365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.067098 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.067071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a853848e-1449-4ae2-834b-912322c73848-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.067254 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.067239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a853848e-1449-4ae2-834b-912322c73848-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.073387 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.073365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vtn7\" (UniqueName: \"kubernetes.io/projected/a853848e-1449-4ae2-834b-912322c73848-kube-api-access-2vtn7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-wbqvj\" (UID: \"a853848e-1449-4ae2-834b-912322c73848\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.227649 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.227570    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:48:59.353156 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:48:59.353128    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"]
Jul 30 16:48:59.355034 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:48:59.355004    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda853848e_1449_4ae2_834b_912322c73848.slice/crio-6c56fc42179ba1cce0de2cc63f58150b995dda534e841642b4d3c6cfbad24092 WatchSource:0}: Error finding container 6c56fc42179ba1cce0de2cc63f58150b995dda534e841642b4d3c6cfbad24092: Status 404 returned error can't find the container with id 6c56fc42179ba1cce0de2cc63f58150b995dda534e841642b4d3c6cfbad24092
Jul 30 16:49:00.259302 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:49:00.259267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj" event={"ID":"a853848e-1449-4ae2-834b-912322c73848","Type":"ContainerStarted","Data":"e629cde59828d4a6148a305735a57d907018f1700e118b71e67ac7db24c8abe1"}
Jul 30 16:49:00.259753 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:49:00.259319    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj" event={"ID":"a853848e-1449-4ae2-834b-912322c73848","Type":"ContainerStarted","Data":"6c56fc42179ba1cce0de2cc63f58150b995dda534e841642b4d3c6cfbad24092"}
Jul 30 16:49:00.278630 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:49:00.278557    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj" podStartSLOduration=2.093899042 podStartE2EDuration="2.27854395s" totalImagesPullingTime="184.644908ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:49:00.277981698 +0000 UTC m=+767.880409324" watchObservedRunningTime="2026-07-30 16:49:00.27854395 +0000 UTC m=+767.880971575"
Jul 30 16:49:09.228299 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:49:09.228257    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:49:09.228868 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:49:09.228326    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:49:09.240455 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:49:09.240424    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:49:09.310428 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:49:09.310397    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-wbqvj"
Jul 30 16:51:41.187308 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.187277    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f6d974979-hljgq"]
Jul 30 16:51:41.187858 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.187462    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f6d974979-hljgq" podUID="31a102a4-f550-4add-b3d2-4945c93f8695" containerName="manager" containerID="cri-o://694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac" gracePeriod=30
Jul 30 16:51:41.429291 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.429265    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:51:41.550926 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.550862    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31a102a4-f550-4add-b3d2-4945c93f8695-webhook-cert\") pod \"31a102a4-f550-4add-b3d2-4945c93f8695\" (UID: \"31a102a4-f550-4add-b3d2-4945c93f8695\") "
Jul 30 16:51:41.551046 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.550935    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kjhm\" (UniqueName: \"kubernetes.io/projected/31a102a4-f550-4add-b3d2-4945c93f8695-kube-api-access-9kjhm\") pod \"31a102a4-f550-4add-b3d2-4945c93f8695\" (UID: \"31a102a4-f550-4add-b3d2-4945c93f8695\") "
Jul 30 16:51:41.552671 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.552643    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a102a4-f550-4add-b3d2-4945c93f8695-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "31a102a4-f550-4add-b3d2-4945c93f8695" (UID: "31a102a4-f550-4add-b3d2-4945c93f8695"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:51:41.552815 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.552801    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31a102a4-f550-4add-b3d2-4945c93f8695-kube-api-access-9kjhm" (OuterVolumeSpecName: "kube-api-access-9kjhm") pod "31a102a4-f550-4add-b3d2-4945c93f8695" (UID: "31a102a4-f550-4add-b3d2-4945c93f8695"). InnerVolumeSpecName "kube-api-access-9kjhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:51:41.651936 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.651911    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31a102a4-f550-4add-b3d2-4945c93f8695-webhook-cert\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:51:41.651936 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.651933    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kjhm\" (UniqueName: \"kubernetes.io/projected/31a102a4-f550-4add-b3d2-4945c93f8695-kube-api-access-9kjhm\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:51:41.947561 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.947537    2569 generic.go:356] "Generic (PLEG): container finished" podID="31a102a4-f550-4add-b3d2-4945c93f8695" containerID="694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac" exitCode=0
Jul 30 16:51:41.947680 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.947577    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-hljgq"
Jul 30 16:51:41.947680 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.947625    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-hljgq" event={"ID":"31a102a4-f550-4add-b3d2-4945c93f8695","Type":"ContainerDied","Data":"694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac"}
Jul 30 16:51:41.947680 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.947669    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-hljgq" event={"ID":"31a102a4-f550-4add-b3d2-4945c93f8695","Type":"ContainerDied","Data":"a711a66019674e7ceaa373a2f4c586d13756b9d6381c5b1bff04eb8e483a41d3"}
Jul 30 16:51:41.947779 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.947690    2569 scope.go:117] "RemoveContainer" containerID="694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac"
Jul 30 16:51:41.962834 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.962813    2569 scope.go:117] "RemoveContainer" containerID="694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac"
Jul 30 16:51:41.963146 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:51:41.963124    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac\": container with ID starting with 694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac not found: ID does not exist" containerID="694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac"
Jul 30 16:51:41.963208 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.963155    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac"} err="failed to get container status \"694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac\": rpc error: code = NotFound desc = could not find container \"694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac\": container with ID starting with 694f6291b7d095dcb7b2403bf93a58033fe107cc3e45c666550a1ed2a99004ac not found: ID does not exist"
Jul 30 16:51:41.974383 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.974358    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f6d974979-hljgq"]
Jul 30 16:51:41.981072 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:41.981047    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f6d974979-hljgq"]
Jul 30 16:51:43.000755 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:51:43.000728    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31a102a4-f550-4add-b3d2-4945c93f8695" path="/var/lib/kubelet/pods/31a102a4-f550-4add-b3d2-4945c93f8695/volumes"
Jul 30 16:52:42.791262 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.791219    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f6d974979-xdjkx"]
Jul 30 16:52:42.791758 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.791637    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31a102a4-f550-4add-b3d2-4945c93f8695" containerName="manager"
Jul 30 16:52:42.791758 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.791655    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="31a102a4-f550-4add-b3d2-4945c93f8695" containerName="manager"
Jul 30 16:52:42.791758 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.791716    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="31a102a4-f550-4add-b3d2-4945c93f8695" containerName="manager"
Jul 30 16:52:42.794919 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.794900    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:42.798733 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.798705    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 16:52:42.798888 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.798710    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 16:52:42.798888 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.798710    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mww4n\""
Jul 30 16:52:42.808507 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.808477    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f6d974979-xdjkx"]
Jul 30 16:52:42.823313 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.823289    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eeda4e1d-91a6-4655-81c4-10a300592727-webhook-cert\") pod \"maas-controller-5f6d974979-xdjkx\" (UID: \"eeda4e1d-91a6-4655-81c4-10a300592727\") " pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:42.823485 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.823336    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdvfj\" (UniqueName: \"kubernetes.io/projected/eeda4e1d-91a6-4655-81c4-10a300592727-kube-api-access-jdvfj\") pod \"maas-controller-5f6d974979-xdjkx\" (UID: \"eeda4e1d-91a6-4655-81c4-10a300592727\") " pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:42.923909 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.923887    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eeda4e1d-91a6-4655-81c4-10a300592727-webhook-cert\") pod \"maas-controller-5f6d974979-xdjkx\" (UID: \"eeda4e1d-91a6-4655-81c4-10a300592727\") " pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:42.924090 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.923935    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdvfj\" (UniqueName: \"kubernetes.io/projected/eeda4e1d-91a6-4655-81c4-10a300592727-kube-api-access-jdvfj\") pod \"maas-controller-5f6d974979-xdjkx\" (UID: \"eeda4e1d-91a6-4655-81c4-10a300592727\") " pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:42.926129 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.926104    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eeda4e1d-91a6-4655-81c4-10a300592727-webhook-cert\") pod \"maas-controller-5f6d974979-xdjkx\" (UID: \"eeda4e1d-91a6-4655-81c4-10a300592727\") " pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:42.933771 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:42.933749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdvfj\" (UniqueName: \"kubernetes.io/projected/eeda4e1d-91a6-4655-81c4-10a300592727-kube-api-access-jdvfj\") pod \"maas-controller-5f6d974979-xdjkx\" (UID: \"eeda4e1d-91a6-4655-81c4-10a300592727\") " pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:43.107037 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:43.106974    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:43.238427 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:43.238400    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f6d974979-xdjkx"]
Jul 30 16:52:43.240157 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:52:43.240132    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeda4e1d_91a6_4655_81c4_10a300592727.slice/crio-3dbaa70be9183bb09fe18a2a836687a1a91576934a5f1c5de761a8ee9b6acb0a WatchSource:0}: Error finding container 3dbaa70be9183bb09fe18a2a836687a1a91576934a5f1c5de761a8ee9b6acb0a: Status 404 returned error can't find the container with id 3dbaa70be9183bb09fe18a2a836687a1a91576934a5f1c5de761a8ee9b6acb0a
Jul 30 16:52:44.224354 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:44.224322    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-xdjkx" event={"ID":"eeda4e1d-91a6-4655-81c4-10a300592727","Type":"ContainerStarted","Data":"bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840"}
Jul 30 16:52:44.224781 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:44.224384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-xdjkx" event={"ID":"eeda4e1d-91a6-4655-81c4-10a300592727","Type":"ContainerStarted","Data":"3dbaa70be9183bb09fe18a2a836687a1a91576934a5f1c5de761a8ee9b6acb0a"}
Jul 30 16:52:44.224781 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:44.224414    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:44.245828 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:44.245784    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f6d974979-xdjkx" podStartSLOduration=1.753735468 podStartE2EDuration="2.245768917s" totalImagesPullingTime="492.033449ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:52:44.243227422 +0000 UTC m=+991.845655046" watchObservedRunningTime="2026-07-30 16:52:44.245768917 +0000 UTC m=+991.848196540"
Jul 30 16:52:55.231053 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:55.231015    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:52:59.869832 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:59.869790    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f6d974979-xdjkx"]
Jul 30 16:52:59.870290 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:52:59.869973    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f6d974979-xdjkx" podUID="eeda4e1d-91a6-4655-81c4-10a300592727" containerName="manager" containerID="cri-o://bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840" gracePeriod=30
Jul 30 16:53:00.116489 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.116469    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:53:00.264255 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.264225    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdvfj\" (UniqueName: \"kubernetes.io/projected/eeda4e1d-91a6-4655-81c4-10a300592727-kube-api-access-jdvfj\") pod \"eeda4e1d-91a6-4655-81c4-10a300592727\" (UID: \"eeda4e1d-91a6-4655-81c4-10a300592727\") "
Jul 30 16:53:00.264414 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.264330    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eeda4e1d-91a6-4655-81c4-10a300592727-webhook-cert\") pod \"eeda4e1d-91a6-4655-81c4-10a300592727\" (UID: \"eeda4e1d-91a6-4655-81c4-10a300592727\") "
Jul 30 16:53:00.266222 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.266192    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eeda4e1d-91a6-4655-81c4-10a300592727-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eeda4e1d-91a6-4655-81c4-10a300592727" (UID: "eeda4e1d-91a6-4655-81c4-10a300592727"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 16:53:00.266323 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.266191    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeda4e1d-91a6-4655-81c4-10a300592727-kube-api-access-jdvfj" (OuterVolumeSpecName: "kube-api-access-jdvfj") pod "eeda4e1d-91a6-4655-81c4-10a300592727" (UID: "eeda4e1d-91a6-4655-81c4-10a300592727"). InnerVolumeSpecName "kube-api-access-jdvfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 16:53:00.305387 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.305360    2569 generic.go:356] "Generic (PLEG): container finished" podID="eeda4e1d-91a6-4655-81c4-10a300592727" containerID="bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840" exitCode=0
Jul 30 16:53:00.305494 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.305416    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-xdjkx"
Jul 30 16:53:00.305494 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.305449    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-xdjkx" event={"ID":"eeda4e1d-91a6-4655-81c4-10a300592727","Type":"ContainerDied","Data":"bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840"}
Jul 30 16:53:00.305566 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.305522    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-xdjkx" event={"ID":"eeda4e1d-91a6-4655-81c4-10a300592727","Type":"ContainerDied","Data":"3dbaa70be9183bb09fe18a2a836687a1a91576934a5f1c5de761a8ee9b6acb0a"}
Jul 30 16:53:00.305566 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.305544    2569 scope.go:117] "RemoveContainer" containerID="bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840"
Jul 30 16:53:00.313568 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.313550    2569 scope.go:117] "RemoveContainer" containerID="bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840"
Jul 30 16:53:00.313890 ip-10-0-135-111 kubenswrapper[2569]: E0730 16:53:00.313815    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840\": container with ID starting with bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840 not found: ID does not exist" containerID="bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840"
Jul 30 16:53:00.313890 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.313840    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840"} err="failed to get container status \"bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840\": rpc error: code = NotFound desc = could not find container \"bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840\": container with ID starting with bfd061a70a6b0cdf2619a09598db9705ae5c1420a574d4be76ee314aaecca840 not found: ID does not exist"
Jul 30 16:53:00.327160 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.327114    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f6d974979-xdjkx"]
Jul 30 16:53:00.329417 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.329393    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f6d974979-xdjkx"]
Jul 30 16:53:00.365786 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.365762    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eeda4e1d-91a6-4655-81c4-10a300592727-webhook-cert\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:53:00.365786 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:00.365784    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdvfj\" (UniqueName: \"kubernetes.io/projected/eeda4e1d-91a6-4655-81c4-10a300592727-kube-api-access-jdvfj\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 16:53:01.000827 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.000798    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeda4e1d-91a6-4655-81c4-10a300592727" path="/var/lib/kubelet/pods/eeda4e1d-91a6-4655-81c4-10a300592727/volumes"
Jul 30 16:53:01.137334 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.137308    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f6d974979-cdhxs"]
Jul 30 16:53:01.137667 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.137654    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeda4e1d-91a6-4655-81c4-10a300592727" containerName="manager"
Jul 30 16:53:01.137723 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.137669    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeda4e1d-91a6-4655-81c4-10a300592727" containerName="manager"
Jul 30 16:53:01.137758 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.137741    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeda4e1d-91a6-4655-81c4-10a300592727" containerName="manager"
Jul 30 16:53:01.141664 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.141646    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 16:53:01.144583 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.144562    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 16:53:01.145283 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.145265    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mww4n\""
Jul 30 16:53:01.145579 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.145559    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 16:53:01.153024 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.152998    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f6d974979-cdhxs"]
Jul 30 16:53:01.272125 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.272068    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c228f2f1-c36f-4bc3-ac18-509574700266-webhook-cert\") pod \"maas-controller-5f6d974979-cdhxs\" (UID: \"c228f2f1-c36f-4bc3-ac18-509574700266\") " pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 16:53:01.272125 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.272105    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsbxv\" (UniqueName: \"kubernetes.io/projected/c228f2f1-c36f-4bc3-ac18-509574700266-kube-api-access-xsbxv\") pod \"maas-controller-5f6d974979-cdhxs\" (UID: \"c228f2f1-c36f-4bc3-ac18-509574700266\") " pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 16:53:01.372954 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.372928    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c228f2f1-c36f-4bc3-ac18-509574700266-webhook-cert\") pod \"maas-controller-5f6d974979-cdhxs\" (UID: \"c228f2f1-c36f-4bc3-ac18-509574700266\") " pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 16:53:01.373055 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.372966    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsbxv\" (UniqueName: \"kubernetes.io/projected/c228f2f1-c36f-4bc3-ac18-509574700266-kube-api-access-xsbxv\") pod \"maas-controller-5f6d974979-cdhxs\" (UID: \"c228f2f1-c36f-4bc3-ac18-509574700266\") " pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 16:53:01.375172 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.375145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c228f2f1-c36f-4bc3-ac18-509574700266-webhook-cert\") pod \"maas-controller-5f6d974979-cdhxs\" (UID: \"c228f2f1-c36f-4bc3-ac18-509574700266\") " pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 16:53:01.381550 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.381532    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsbxv\" (UniqueName: \"kubernetes.io/projected/c228f2f1-c36f-4bc3-ac18-509574700266-kube-api-access-xsbxv\") pod \"maas-controller-5f6d974979-cdhxs\" (UID: \"c228f2f1-c36f-4bc3-ac18-509574700266\") " pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 16:53:01.452743 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.452721    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 16:53:01.580388 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.580361    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f6d974979-cdhxs"]
Jul 30 16:53:01.582804 ip-10-0-135-111 kubenswrapper[2569]: W0730 16:53:01.582779    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc228f2f1_c36f_4bc3_ac18_509574700266.slice/crio-70ba3522b0659ef433e86fc093d63127ad633393d5926139773a71648c1f42f5 WatchSource:0}: Error finding container 70ba3522b0659ef433e86fc093d63127ad633393d5926139773a71648c1f42f5: Status 404 returned error can't find the container with id 70ba3522b0659ef433e86fc093d63127ad633393d5926139773a71648c1f42f5
Jul 30 16:53:01.584209 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:01.584193    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 16:53:02.316220 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:02.316187    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-cdhxs" event={"ID":"c228f2f1-c36f-4bc3-ac18-509574700266","Type":"ContainerStarted","Data":"45d11b3ee23dc522b358a49ef83695b59e2cb54a0b6d14a77d2e7952ec029bfa"}
Jul 30 16:53:02.316568 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:02.316239    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f6d974979-cdhxs" event={"ID":"c228f2f1-c36f-4bc3-ac18-509574700266","Type":"ContainerStarted","Data":"70ba3522b0659ef433e86fc093d63127ad633393d5926139773a71648c1f42f5"}
Jul 30 16:53:02.316568 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:02.316266    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 16:53:02.337831 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:02.337787    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f6d974979-cdhxs" podStartSLOduration=1.04858248 podStartE2EDuration="1.337772819s" totalImagesPullingTime="289.190339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 16:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 16:53:02.336059482 +0000 UTC m=+1009.938487107" watchObservedRunningTime="2026-07-30 16:53:02.337772819 +0000 UTC m=+1009.940200444"
Jul 30 16:53:13.323187 ip-10-0-135-111 kubenswrapper[2569]: I0730 16:53:13.323101    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f6d974979-cdhxs"
Jul 30 17:10:18.134902 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.134869    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"]
Jul 30 17:10:18.138095 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.138077    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.140620 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.140590    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-36c7a10f-serving-cert\""
Jul 30 17:10:18.148904 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.148878    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"]
Jul 30 17:10:18.196283 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.196244    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-maas-api-tls\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.196424 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.196323    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-openshift-service-ca\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.196424 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.196389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlnft\" (UniqueName: \"kubernetes.io/projected/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-kube-api-access-jlnft\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.296908 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.296886    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlnft\" (UniqueName: \"kubernetes.io/projected/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-kube-api-access-jlnft\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.297011 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.296955    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-maas-api-tls\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.297011 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.296990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-openshift-service-ca\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.297107 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:10:18.297089    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-serving-cert: secret "maas-api-e2e-ait-36c7a10f-serving-cert" not found
Jul 30 17:10:18.297160 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:10:18.297150    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-maas-api-tls podName:1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5 nodeName:}" failed. No retries permitted until 2026-07-30 17:10:18.79713382 +0000 UTC m=+2046.399561423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-maas-api-tls") pod "maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl" (UID: "1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5") : secret "maas-api-e2e-ait-36c7a10f-serving-cert" not found
Jul 30 17:10:18.297543 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.297527    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-openshift-service-ca\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.305423 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.305403    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlnft\" (UniqueName: \"kubernetes.io/projected/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-kube-api-access-jlnft\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.801815 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.801780    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-maas-api-tls\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.803970 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.803943    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-maas-api-tls\") pod \"maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:18.837977 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.837945    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"]
Jul 30 17:10:18.841374 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.841356    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:18.844414 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.844391    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-36c7a10f-dockercfg-gp8pt\""
Jul 30 17:10:18.844526 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.844421    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-36c7a10f\""
Jul 30 17:10:18.850883 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.850862    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"]
Jul 30 17:10:18.885764 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.885745    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"]
Jul 30 17:10:18.888844 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.888830    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:18.901811 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.901791    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"]
Jul 30 17:10:18.902394 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.902376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjmps\" (UniqueName: \"kubernetes.io/projected/ac8df002-5d06-4507-8c93-4205881dabca-kube-api-access-sjmps\") pod \"payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr\" (UID: \"ac8df002-5d06-4507-8c93-4205881dabca\") " pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:18.902488 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.902439    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqb89\" (UniqueName: \"kubernetes.io/projected/2797b735-780f-40e1-b3a7-9543f340229f-kube-api-access-dqb89\") pod \"payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st\" (UID: \"2797b735-780f-40e1-b3a7-9543f340229f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:18.902550 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.902515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac8df002-5d06-4507-8c93-4205881dabca-plugins-config-volume\") pod \"payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr\" (UID: \"ac8df002-5d06-4507-8c93-4205881dabca\") " pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:18.902638 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:18.902563    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2797b735-780f-40e1-b3a7-9543f340229f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st\" (UID: \"2797b735-780f-40e1-b3a7-9543f340229f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:19.003401 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.003363    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqb89\" (UniqueName: \"kubernetes.io/projected/2797b735-780f-40e1-b3a7-9543f340229f-kube-api-access-dqb89\") pod \"payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st\" (UID: \"2797b735-780f-40e1-b3a7-9543f340229f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:19.003689 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.003663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac8df002-5d06-4507-8c93-4205881dabca-plugins-config-volume\") pod \"payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr\" (UID: \"ac8df002-5d06-4507-8c93-4205881dabca\") " pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:19.003790 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.003764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2797b735-780f-40e1-b3a7-9543f340229f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st\" (UID: \"2797b735-780f-40e1-b3a7-9543f340229f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:19.003887 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.003871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjmps\" (UniqueName: \"kubernetes.io/projected/ac8df002-5d06-4507-8c93-4205881dabca-kube-api-access-sjmps\") pod \"payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr\" (UID: \"ac8df002-5d06-4507-8c93-4205881dabca\") " pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:19.004336 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.004315    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac8df002-5d06-4507-8c93-4205881dabca-plugins-config-volume\") pod \"payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr\" (UID: \"ac8df002-5d06-4507-8c93-4205881dabca\") " pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:19.004416 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.004383    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2797b735-780f-40e1-b3a7-9543f340229f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st\" (UID: \"2797b735-780f-40e1-b3a7-9543f340229f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:19.012925 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.012901    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqb89\" (UniqueName: \"kubernetes.io/projected/2797b735-780f-40e1-b3a7-9543f340229f-kube-api-access-dqb89\") pod \"payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st\" (UID: \"2797b735-780f-40e1-b3a7-9543f340229f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:19.012925 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.012919    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjmps\" (UniqueName: \"kubernetes.io/projected/ac8df002-5d06-4507-8c93-4205881dabca-kube-api-access-sjmps\") pod \"payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr\" (UID: \"ac8df002-5d06-4507-8c93-4205881dabca\") " pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:19.048618 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.048581    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:19.152923 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.152890    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:19.178377 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.178355    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"]
Jul 30 17:10:19.180642 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:10:19.180596    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3c1114_d7da_46d2_9cc9_1d1f7e1526f5.slice/crio-d33abba7b4f623cb57b843e06149e4801463b331049f2239807cd154ef8d0b12 WatchSource:0}: Error finding container d33abba7b4f623cb57b843e06149e4801463b331049f2239807cd154ef8d0b12: Status 404 returned error can't find the container with id d33abba7b4f623cb57b843e06149e4801463b331049f2239807cd154ef8d0b12
Jul 30 17:10:19.182490 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.182472    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:10:19.197709 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.197691    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:19.289862 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.289408    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"]
Jul 30 17:10:19.292764 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:10:19.292735    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2797b735_780f_40e1_b3a7_9543f340229f.slice/crio-b32018483e5bee536bd98556d86725a0b03bcd579ff9f759ca9b4f21726384d3 WatchSource:0}: Error finding container b32018483e5bee536bd98556d86725a0b03bcd579ff9f759ca9b4f21726384d3: Status 404 returned error can't find the container with id b32018483e5bee536bd98556d86725a0b03bcd579ff9f759ca9b4f21726384d3
Jul 30 17:10:19.349407 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.349378    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"]
Jul 30 17:10:19.351668 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:10:19.351640    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac8df002_5d06_4507_8c93_4205881dabca.slice/crio-d4a8e6eb098c9d1fe76fdd1f9802616dc4760578ca8bfabdcadc45a3f052ecde WatchSource:0}: Error finding container d4a8e6eb098c9d1fe76fdd1f9802616dc4760578ca8bfabdcadc45a3f052ecde: Status 404 returned error can't find the container with id d4a8e6eb098c9d1fe76fdd1f9802616dc4760578ca8bfabdcadc45a3f052ecde
Jul 30 17:10:19.731007 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.730972    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl" event={"ID":"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5","Type":"ContainerStarted","Data":"d33abba7b4f623cb57b843e06149e4801463b331049f2239807cd154ef8d0b12"}
Jul 30 17:10:19.732434 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.732400    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr" event={"ID":"ac8df002-5d06-4507-8c93-4205881dabca","Type":"ContainerStarted","Data":"a2819bf5efafaaa32906229c6689b68b8a96ccc519625b6befd4facea988f27d"}
Jul 30 17:10:19.732584 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.732453    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr" event={"ID":"ac8df002-5d06-4507-8c93-4205881dabca","Type":"ContainerStarted","Data":"d4a8e6eb098c9d1fe76fdd1f9802616dc4760578ca8bfabdcadc45a3f052ecde"}
Jul 30 17:10:19.732584 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.732565    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:19.733801 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.733779    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st" event={"ID":"2797b735-780f-40e1-b3a7-9543f340229f","Type":"ContainerStarted","Data":"196b318975ba38abb6e87437e48e63b2ca8dc09991fbd0a02485d0757bbaabb9"}
Jul 30 17:10:19.733934 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.733815    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st" event={"ID":"2797b735-780f-40e1-b3a7-9543f340229f","Type":"ContainerStarted","Data":"b32018483e5bee536bd98556d86725a0b03bcd579ff9f759ca9b4f21726384d3"}
Jul 30 17:10:19.733934 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.733835    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:19.752626 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.752567    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr" podStartSLOduration=1.7525499519999999 podStartE2EDuration="1.752549952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:10:19.750830587 +0000 UTC m=+2047.353258211" watchObservedRunningTime="2026-07-30 17:10:19.752549952 +0000 UTC m=+2047.354977581"
Jul 30 17:10:19.769396 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:19.769356    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st" podStartSLOduration=1.769344068 podStartE2EDuration="1.769344068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:10:19.76871181 +0000 UTC m=+2047.371139436" watchObservedRunningTime="2026-07-30 17:10:19.769344068 +0000 UTC m=+2047.371771694"
Jul 30 17:10:20.739694 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:20.739660    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl" event={"ID":"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5","Type":"ContainerStarted","Data":"676b22537e86fb702e99b2672f92b7a639c2eea282314e24fbb1b63754f88b09"}
Jul 30 17:10:20.740151 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:20.739759    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:20.758269 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:20.758226    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl" podStartSLOduration=2.079805448 podStartE2EDuration="2.758213568s" totalImagesPullingTime="678.40812ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:10:20.756794841 +0000 UTC m=+2048.359222469" watchObservedRunningTime="2026-07-30 17:10:20.758213568 +0000 UTC m=+2048.360641228"
Jul 30 17:10:26.750067 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:26.750036    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:30.742228 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:30.742194    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:30.742667 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:30.742264    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:58.646849 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.646811    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"]
Jul 30 17:10:58.647402 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.647052    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl" podUID="1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5" containerName="maas-api" containerID="cri-o://676b22537e86fb702e99b2672f92b7a639c2eea282314e24fbb1b63754f88b09" gracePeriod=30
Jul 30 17:10:58.699558 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.699522    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"]
Jul 30 17:10:58.699814 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.699769    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr" podUID="ac8df002-5d06-4507-8c93-4205881dabca" containerName="payload-processing" containerID="cri-o://a2819bf5efafaaa32906229c6689b68b8a96ccc519625b6befd4facea988f27d" gracePeriod=30
Jul 30 17:10:58.708353 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.708327    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"]
Jul 30 17:10:58.708582 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.708556    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st" podUID="2797b735-780f-40e1-b3a7-9543f340229f" containerName="payload-pre-processing" containerID="cri-o://196b318975ba38abb6e87437e48e63b2ca8dc09991fbd0a02485d0757bbaabb9" gracePeriod=30
Jul 30 17:10:58.807776 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:10:58.807744    2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac8df002_5d06_4507_8c93_4205881dabca.slice/crio-conmon-a2819bf5efafaaa32906229c6689b68b8a96ccc519625b6befd4facea988f27d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac8df002_5d06_4507_8c93_4205881dabca.slice/crio-a2819bf5efafaaa32906229c6689b68b8a96ccc519625b6befd4facea988f27d.scope\": RecentStats: unable to find data in memory cache]"
Jul 30 17:10:58.915909 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.915714    2569 generic.go:356] "Generic (PLEG): container finished" podID="2797b735-780f-40e1-b3a7-9543f340229f" containerID="196b318975ba38abb6e87437e48e63b2ca8dc09991fbd0a02485d0757bbaabb9" exitCode=0
Jul 30 17:10:58.915909 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.915845    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st" event={"ID":"2797b735-780f-40e1-b3a7-9543f340229f","Type":"ContainerDied","Data":"196b318975ba38abb6e87437e48e63b2ca8dc09991fbd0a02485d0757bbaabb9"}
Jul 30 17:10:58.918207 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.918053    2569 generic.go:356] "Generic (PLEG): container finished" podID="1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5" containerID="676b22537e86fb702e99b2672f92b7a639c2eea282314e24fbb1b63754f88b09" exitCode=0
Jul 30 17:10:58.918207 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.918141    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl" event={"ID":"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5","Type":"ContainerDied","Data":"676b22537e86fb702e99b2672f92b7a639c2eea282314e24fbb1b63754f88b09"}
Jul 30 17:10:58.920361 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.920312    2569 generic.go:356] "Generic (PLEG): container finished" podID="ac8df002-5d06-4507-8c93-4205881dabca" containerID="a2819bf5efafaaa32906229c6689b68b8a96ccc519625b6befd4facea988f27d" exitCode=0
Jul 30 17:10:58.920468 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.920434    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr" event={"ID":"ac8df002-5d06-4507-8c93-4205881dabca","Type":"ContainerDied","Data":"a2819bf5efafaaa32906229c6689b68b8a96ccc519625b6befd4facea988f27d"}
Jul 30 17:10:58.987297 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.987273    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:58.990526 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:58.990507    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:59.024098 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.024024    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:59.135258 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.135233    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjmps\" (UniqueName: \"kubernetes.io/projected/ac8df002-5d06-4507-8c93-4205881dabca-kube-api-access-sjmps\") pod \"ac8df002-5d06-4507-8c93-4205881dabca\" (UID: \"ac8df002-5d06-4507-8c93-4205881dabca\") "
Jul 30 17:10:59.135415 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.135271    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2797b735-780f-40e1-b3a7-9543f340229f-plugins-config-volume\") pod \"2797b735-780f-40e1-b3a7-9543f340229f\" (UID: \"2797b735-780f-40e1-b3a7-9543f340229f\") "
Jul 30 17:10:59.135415 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.135325    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqb89\" (UniqueName: \"kubernetes.io/projected/2797b735-780f-40e1-b3a7-9543f340229f-kube-api-access-dqb89\") pod \"2797b735-780f-40e1-b3a7-9543f340229f\" (UID: \"2797b735-780f-40e1-b3a7-9543f340229f\") "
Jul 30 17:10:59.135415 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.135391    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlnft\" (UniqueName: \"kubernetes.io/projected/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-kube-api-access-jlnft\") pod \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") "
Jul 30 17:10:59.135626 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.135583    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-maas-api-tls\") pod \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") "
Jul 30 17:10:59.135696 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.135676    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-openshift-service-ca\") pod \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\" (UID: \"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5\") "
Jul 30 17:10:59.135766 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.135708    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac8df002-5d06-4507-8c93-4205881dabca-plugins-config-volume\") pod \"ac8df002-5d06-4507-8c93-4205881dabca\" (UID: \"ac8df002-5d06-4507-8c93-4205881dabca\") "
Jul 30 17:10:59.135817 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.135788    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2797b735-780f-40e1-b3a7-9543f340229f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2797b735-780f-40e1-b3a7-9543f340229f" (UID: "2797b735-780f-40e1-b3a7-9543f340229f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:10:59.136062 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.136042    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2797b735-780f-40e1-b3a7-9543f340229f-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:10:59.136328 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.136297    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5" (UID: "1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:10:59.136417 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.136328    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac8df002-5d06-4507-8c93-4205881dabca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac8df002-5d06-4507-8c93-4205881dabca" (UID: "ac8df002-5d06-4507-8c93-4205881dabca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:10:59.137763 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.137727    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2797b735-780f-40e1-b3a7-9543f340229f-kube-api-access-dqb89" (OuterVolumeSpecName: "kube-api-access-dqb89") pod "2797b735-780f-40e1-b3a7-9543f340229f" (UID: "2797b735-780f-40e1-b3a7-9543f340229f"). InnerVolumeSpecName "kube-api-access-dqb89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:10:59.137763 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.137743    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac8df002-5d06-4507-8c93-4205881dabca-kube-api-access-sjmps" (OuterVolumeSpecName: "kube-api-access-sjmps") pod "ac8df002-5d06-4507-8c93-4205881dabca" (UID: "ac8df002-5d06-4507-8c93-4205881dabca"). InnerVolumeSpecName "kube-api-access-sjmps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:10:59.138075 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.138040    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-kube-api-access-jlnft" (OuterVolumeSpecName: "kube-api-access-jlnft") pod "1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5" (UID: "1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5"). InnerVolumeSpecName "kube-api-access-jlnft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:10:59.138488 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.138465    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5" (UID: "1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:10:59.236413 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.236364    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjmps\" (UniqueName: \"kubernetes.io/projected/ac8df002-5d06-4507-8c93-4205881dabca-kube-api-access-sjmps\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:10:59.236413 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.236384    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqb89\" (UniqueName: \"kubernetes.io/projected/2797b735-780f-40e1-b3a7-9543f340229f-kube-api-access-dqb89\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:10:59.236413 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.236394    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlnft\" (UniqueName: \"kubernetes.io/projected/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-kube-api-access-jlnft\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:10:59.236413 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.236402    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:10:59.236413 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.236410    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:10:59.236641 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.236418    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac8df002-5d06-4507-8c93-4205881dabca-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:10:59.926025 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.925959    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"
Jul 30 17:10:59.926025 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.925972    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl" event={"ID":"1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5","Type":"ContainerDied","Data":"d33abba7b4f623cb57b843e06149e4801463b331049f2239807cd154ef8d0b12"}
Jul 30 17:10:59.926495 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.926036    2569 scope.go:117] "RemoveContainer" containerID="676b22537e86fb702e99b2672f92b7a639c2eea282314e24fbb1b63754f88b09"
Jul 30 17:10:59.927406 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.927389    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"
Jul 30 17:10:59.927495 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.927384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr" event={"ID":"ac8df002-5d06-4507-8c93-4205881dabca","Type":"ContainerDied","Data":"d4a8e6eb098c9d1fe76fdd1f9802616dc4760578ca8bfabdcadc45a3f052ecde"}
Jul 30 17:10:59.928825 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.928809    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"
Jul 30 17:10:59.928825 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.928813    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st" event={"ID":"2797b735-780f-40e1-b3a7-9543f340229f","Type":"ContainerDied","Data":"b32018483e5bee536bd98556d86725a0b03bcd579ff9f759ca9b4f21726384d3"}
Jul 30 17:10:59.936961 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.936942    2569 scope.go:117] "RemoveContainer" containerID="a2819bf5efafaaa32906229c6689b68b8a96ccc519625b6befd4facea988f27d"
Jul 30 17:10:59.945343 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.945327    2569 scope.go:117] "RemoveContainer" containerID="196b318975ba38abb6e87437e48e63b2ca8dc09991fbd0a02485d0757bbaabb9"
Jul 30 17:10:59.959307 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.959286    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"]
Jul 30 17:10:59.965308 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.965288    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-36c7a10f-f6dfc5bc4-xvflr"]
Jul 30 17:10:59.981834 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.981812    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"]
Jul 30 17:10:59.986133 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.986114    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-36c7a10f-95d45b569-xnrzl"]
Jul 30 17:10:59.998351 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:10:59.998326    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"]
Jul 30 17:11:00.004468 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:00.004449    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-36c7a10f-5664d8c5cb-vc7st"]
Jul 30 17:11:00.999743 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:00.999710    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5" path="/var/lib/kubelet/pods/1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5/volumes"
Jul 30 17:11:01.000140 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:01.000080    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2797b735-780f-40e1-b3a7-9543f340229f" path="/var/lib/kubelet/pods/2797b735-780f-40e1-b3a7-9543f340229f/volumes"
Jul 30 17:11:01.000411 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:01.000397    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac8df002-5d06-4507-8c93-4205881dabca" path="/var/lib/kubelet/pods/ac8df002-5d06-4507-8c93-4205881dabca/volumes"
Jul 30 17:11:02.544348 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544315    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"]
Jul 30 17:11:02.544751 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544675    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5" containerName="maas-api"
Jul 30 17:11:02.544751 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544691    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5" containerName="maas-api"
Jul 30 17:11:02.544751 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544713    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac8df002-5d06-4507-8c93-4205881dabca" containerName="payload-processing"
Jul 30 17:11:02.544751 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544719    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac8df002-5d06-4507-8c93-4205881dabca" containerName="payload-processing"
Jul 30 17:11:02.544751 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544730    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2797b735-780f-40e1-b3a7-9543f340229f" containerName="payload-pre-processing"
Jul 30 17:11:02.544751 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544736    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2797b735-780f-40e1-b3a7-9543f340229f" containerName="payload-pre-processing"
Jul 30 17:11:02.544949 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544790    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2797b735-780f-40e1-b3a7-9543f340229f" containerName="payload-pre-processing"
Jul 30 17:11:02.544949 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544798    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac8df002-5d06-4507-8c93-4205881dabca" containerName="payload-processing"
Jul 30 17:11:02.544949 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.544803    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d3c1114-d7da-46d2-9cc9-1d1f7e1526f5" containerName="maas-api"
Jul 30 17:11:02.549270 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.549251    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:02.551890 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.551871    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-b68b3510-serving-cert\""
Jul 30 17:11:02.563711 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.563691    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"]
Jul 30 17:11:02.659893 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.659865    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg98c\" (UniqueName: \"kubernetes.io/projected/1f215739-0873-43e7-bdf0-55dc38a599c7-kube-api-access-wg98c\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:02.660006 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.659905    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f215739-0873-43e7-bdf0-55dc38a599c7-maas-api-tls\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:02.660084 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.660042    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f215739-0873-43e7-bdf0-55dc38a599c7-openshift-service-ca\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:02.761256 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.761235    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f215739-0873-43e7-bdf0-55dc38a599c7-openshift-service-ca\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:02.761358 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.761268    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg98c\" (UniqueName: \"kubernetes.io/projected/1f215739-0873-43e7-bdf0-55dc38a599c7-kube-api-access-wg98c\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:02.761358 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.761286    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f215739-0873-43e7-bdf0-55dc38a599c7-maas-api-tls\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:02.761435 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:11:02.761389    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-serving-cert: secret "maas-api-e2e-migrate-b68b3510-serving-cert" not found
Jul 30 17:11:02.761481 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:11:02.761450    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f215739-0873-43e7-bdf0-55dc38a599c7-maas-api-tls podName:1f215739-0873-43e7-bdf0-55dc38a599c7 nodeName:}" failed. No retries permitted until 2026-07-30 17:11:03.261432122 +0000 UTC m=+2090.863859727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f215739-0873-43e7-bdf0-55dc38a599c7-maas-api-tls") pod "maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds" (UID: "1f215739-0873-43e7-bdf0-55dc38a599c7") : secret "maas-api-e2e-migrate-b68b3510-serving-cert" not found
Jul 30 17:11:02.762003 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.761986    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f215739-0873-43e7-bdf0-55dc38a599c7-openshift-service-ca\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:02.770217 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:02.770191    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg98c\" (UniqueName: \"kubernetes.io/projected/1f215739-0873-43e7-bdf0-55dc38a599c7-kube-api-access-wg98c\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:03.218215 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.218183    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"]
Jul 30 17:11:03.221719 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.221700    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:03.224486 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.224464    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-b68b3510-dockercfg-k7hdg\""
Jul 30 17:11:03.226580 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.226555    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b68b3510\""
Jul 30 17:11:03.234108 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.234090    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"]
Jul 30 17:11:03.265582 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.265558    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f215739-0873-43e7-bdf0-55dc38a599c7-maas-api-tls\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:03.267773 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.267757    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f215739-0873-43e7-bdf0-55dc38a599c7-maas-api-tls\") pod \"maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:03.289284 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.289259    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"]
Jul 30 17:11:03.292459 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.292445    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:03.300691 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.300668    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"]
Jul 30 17:11:03.366801 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.366765    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlgt4\" (UniqueName: \"kubernetes.io/projected/de7ad458-04e4-4322-b234-aa69a419adf8-kube-api-access-xlgt4\") pod \"payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z\" (UID: \"de7ad458-04e4-4322-b234-aa69a419adf8\") " pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:03.366949 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.366849    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de7ad458-04e4-4322-b234-aa69a419adf8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z\" (UID: \"de7ad458-04e4-4322-b234-aa69a419adf8\") " pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:03.459280 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.459249    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:03.468198 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.468172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlgt4\" (UniqueName: \"kubernetes.io/projected/de7ad458-04e4-4322-b234-aa69a419adf8-kube-api-access-xlgt4\") pod \"payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z\" (UID: \"de7ad458-04e4-4322-b234-aa69a419adf8\") " pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:03.468338 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.468219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef6364f-7eed-4394-9de1-d6a924110730-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c\" (UID: \"7ef6364f-7eed-4394-9de1-d6a924110730\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:03.468338 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.468244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de7ad458-04e4-4322-b234-aa69a419adf8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z\" (UID: \"de7ad458-04e4-4322-b234-aa69a419adf8\") " pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:03.468338 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.468266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb44l\" (UniqueName: \"kubernetes.io/projected/7ef6364f-7eed-4394-9de1-d6a924110730-kube-api-access-rb44l\") pod \"payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c\" (UID: \"7ef6364f-7eed-4394-9de1-d6a924110730\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:03.468918 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.468896    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de7ad458-04e4-4322-b234-aa69a419adf8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z\" (UID: \"de7ad458-04e4-4322-b234-aa69a419adf8\") " pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:03.480157 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.480131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlgt4\" (UniqueName: \"kubernetes.io/projected/de7ad458-04e4-4322-b234-aa69a419adf8-kube-api-access-xlgt4\") pod \"payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z\" (UID: \"de7ad458-04e4-4322-b234-aa69a419adf8\") " pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:03.532516 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.532488    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:03.569345 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.569311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef6364f-7eed-4394-9de1-d6a924110730-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c\" (UID: \"7ef6364f-7eed-4394-9de1-d6a924110730\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:03.569802 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.569378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb44l\" (UniqueName: \"kubernetes.io/projected/7ef6364f-7eed-4394-9de1-d6a924110730-kube-api-access-rb44l\") pod \"payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c\" (UID: \"7ef6364f-7eed-4394-9de1-d6a924110730\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:03.570188 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.570167    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef6364f-7eed-4394-9de1-d6a924110730-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c\" (UID: \"7ef6364f-7eed-4394-9de1-d6a924110730\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:03.579512 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.579487    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb44l\" (UniqueName: \"kubernetes.io/projected/7ef6364f-7eed-4394-9de1-d6a924110730-kube-api-access-rb44l\") pod \"payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c\" (UID: \"7ef6364f-7eed-4394-9de1-d6a924110730\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:03.596164 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.596007    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"]
Jul 30 17:11:03.601515 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:11:03.601443    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f215739_0873_43e7_bdf0_55dc38a599c7.slice/crio-150dc85a90c20eeeb751b17e1bf69bfbbb471709ac12e08a4cbec1467c2e2b8d WatchSource:0}: Error finding container 150dc85a90c20eeeb751b17e1bf69bfbbb471709ac12e08a4cbec1467c2e2b8d: Status 404 returned error can't find the container with id 150dc85a90c20eeeb751b17e1bf69bfbbb471709ac12e08a4cbec1467c2e2b8d
Jul 30 17:11:03.602160 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.602108    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:03.681120 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.681017    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"]
Jul 30 17:11:03.695403 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:11:03.695368    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde7ad458_04e4_4322_b234_aa69a419adf8.slice/crio-127f451acbc490797d91ca54582b5ac6a13a3b39002752ce8d416550f4b8c8b2 WatchSource:0}: Error finding container 127f451acbc490797d91ca54582b5ac6a13a3b39002752ce8d416550f4b8c8b2: Status 404 returned error can't find the container with id 127f451acbc490797d91ca54582b5ac6a13a3b39002752ce8d416550f4b8c8b2
Jul 30 17:11:03.951755 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.951721    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z" event={"ID":"de7ad458-04e4-4322-b234-aa69a419adf8","Type":"ContainerStarted","Data":"4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41"}
Jul 30 17:11:03.951931 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.951778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z" event={"ID":"de7ad458-04e4-4322-b234-aa69a419adf8","Type":"ContainerStarted","Data":"127f451acbc490797d91ca54582b5ac6a13a3b39002752ce8d416550f4b8c8b2"}
Jul 30 17:11:03.951931 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.951806    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:03.953033 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.953006    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds" event={"ID":"1f215739-0873-43e7-bdf0-55dc38a599c7","Type":"ContainerStarted","Data":"150dc85a90c20eeeb751b17e1bf69bfbbb471709ac12e08a4cbec1467c2e2b8d"}
Jul 30 17:11:03.971807 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.971766    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z" podStartSLOduration=0.971751129 podStartE2EDuration="971.751129ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:11:03.96978421 +0000 UTC m=+2091.572211836" watchObservedRunningTime="2026-07-30 17:11:03.971751129 +0000 UTC m=+2091.574178786"
Jul 30 17:11:03.983925 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:03.983902    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"]
Jul 30 17:11:03.985882 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:11:03.985859    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef6364f_7eed_4394_9de1_d6a924110730.slice/crio-2989cf667eeacd22f8d4d15f70f49c95fda65c5b63a83946edfba6ce2162bd0b WatchSource:0}: Error finding container 2989cf667eeacd22f8d4d15f70f49c95fda65c5b63a83946edfba6ce2162bd0b: Status 404 returned error can't find the container with id 2989cf667eeacd22f8d4d15f70f49c95fda65c5b63a83946edfba6ce2162bd0b
Jul 30 17:11:04.960056 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:04.960019    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds" event={"ID":"1f215739-0873-43e7-bdf0-55dc38a599c7","Type":"ContainerStarted","Data":"7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f"}
Jul 30 17:11:04.960500 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:04.960133    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:04.961656 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:04.961630    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c" event={"ID":"7ef6364f-7eed-4394-9de1-d6a924110730","Type":"ContainerStarted","Data":"f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684"}
Jul 30 17:11:04.961729 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:04.961682    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c" event={"ID":"7ef6364f-7eed-4394-9de1-d6a924110730","Type":"ContainerStarted","Data":"2989cf667eeacd22f8d4d15f70f49c95fda65c5b63a83946edfba6ce2162bd0b"}
Jul 30 17:11:04.979167 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:04.979118    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds" podStartSLOduration=2.587760109 podStartE2EDuration="2.979105208s" totalImagesPullingTime="391.345099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:11:04.977185769 +0000 UTC m=+2092.579613396" watchObservedRunningTime="2026-07-30 17:11:04.979105208 +0000 UTC m=+2092.581532834"
Jul 30 17:11:04.994556 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:04.994516    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c" podStartSLOduration=1.9945032729999999 podStartE2EDuration="1.994503273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:11:04.992396747 +0000 UTC m=+2092.594824372" watchObservedRunningTime="2026-07-30 17:11:04.994503273 +0000 UTC m=+2092.596930897"
Jul 30 17:11:05.966493 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:05.966469    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:10.971724 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:10.971687    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:14.963476 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:14.963417    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:16.972429 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:16.972392    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:34.057274 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.057244    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"]
Jul 30 17:11:34.057837 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.057443    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds" podUID="1f215739-0873-43e7-bdf0-55dc38a599c7" containerName="maas-api" containerID="cri-o://7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f" gracePeriod=30
Jul 30 17:11:34.120552 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.120520    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"]
Jul 30 17:11:34.120840 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.120781    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z" podUID="de7ad458-04e4-4322-b234-aa69a419adf8" containerName="payload-processing" containerID="cri-o://4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41" gracePeriod=30
Jul 30 17:11:34.127323 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.127047    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"]
Jul 30 17:11:34.127323 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.127240    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c" podUID="7ef6364f-7eed-4394-9de1-d6a924110730" containerName="payload-pre-processing" containerID="cri-o://f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684" gracePeriod=30
Jul 30 17:11:34.370920 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.370893    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:34.422013 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.421984    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:34.425189 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.425164    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:34.425373 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.425355    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlgt4\" (UniqueName: \"kubernetes.io/projected/de7ad458-04e4-4322-b234-aa69a419adf8-kube-api-access-xlgt4\") pod \"de7ad458-04e4-4322-b234-aa69a419adf8\" (UID: \"de7ad458-04e4-4322-b234-aa69a419adf8\") "
Jul 30 17:11:34.425460 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.425447    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de7ad458-04e4-4322-b234-aa69a419adf8-plugins-config-volume\") pod \"de7ad458-04e4-4322-b234-aa69a419adf8\" (UID: \"de7ad458-04e4-4322-b234-aa69a419adf8\") "
Jul 30 17:11:34.425820 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.425798    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de7ad458-04e4-4322-b234-aa69a419adf8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de7ad458-04e4-4322-b234-aa69a419adf8" (UID: "de7ad458-04e4-4322-b234-aa69a419adf8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:11:34.428579 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.428559    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de7ad458-04e4-4322-b234-aa69a419adf8-kube-api-access-xlgt4" (OuterVolumeSpecName: "kube-api-access-xlgt4") pod "de7ad458-04e4-4322-b234-aa69a419adf8" (UID: "de7ad458-04e4-4322-b234-aa69a419adf8"). InnerVolumeSpecName "kube-api-access-xlgt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:11:34.526336 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.526303    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg98c\" (UniqueName: \"kubernetes.io/projected/1f215739-0873-43e7-bdf0-55dc38a599c7-kube-api-access-wg98c\") pod \"1f215739-0873-43e7-bdf0-55dc38a599c7\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") "
Jul 30 17:11:34.526464 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.526361    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f215739-0873-43e7-bdf0-55dc38a599c7-maas-api-tls\") pod \"1f215739-0873-43e7-bdf0-55dc38a599c7\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") "
Jul 30 17:11:34.526464 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.526405    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rb44l\" (UniqueName: \"kubernetes.io/projected/7ef6364f-7eed-4394-9de1-d6a924110730-kube-api-access-rb44l\") pod \"7ef6364f-7eed-4394-9de1-d6a924110730\" (UID: \"7ef6364f-7eed-4394-9de1-d6a924110730\") "
Jul 30 17:11:34.526464 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.526456    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f215739-0873-43e7-bdf0-55dc38a599c7-openshift-service-ca\") pod \"1f215739-0873-43e7-bdf0-55dc38a599c7\" (UID: \"1f215739-0873-43e7-bdf0-55dc38a599c7\") "
Jul 30 17:11:34.526649 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.526515    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef6364f-7eed-4394-9de1-d6a924110730-plugins-config-volume\") pod \"7ef6364f-7eed-4394-9de1-d6a924110730\" (UID: \"7ef6364f-7eed-4394-9de1-d6a924110730\") "
Jul 30 17:11:34.526777 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.526755    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlgt4\" (UniqueName: \"kubernetes.io/projected/de7ad458-04e4-4322-b234-aa69a419adf8-kube-api-access-xlgt4\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:11:34.526777 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.526778    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de7ad458-04e4-4322-b234-aa69a419adf8-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:11:34.526963 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.526866    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f215739-0873-43e7-bdf0-55dc38a599c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1f215739-0873-43e7-bdf0-55dc38a599c7" (UID: "1f215739-0873-43e7-bdf0-55dc38a599c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:11:34.526963 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.526883    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ef6364f-7eed-4394-9de1-d6a924110730-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ef6364f-7eed-4394-9de1-d6a924110730" (UID: "7ef6364f-7eed-4394-9de1-d6a924110730"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:11:34.528477 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.528444    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f215739-0873-43e7-bdf0-55dc38a599c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1f215739-0873-43e7-bdf0-55dc38a599c7" (UID: "1f215739-0873-43e7-bdf0-55dc38a599c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:11:34.528566 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.528514    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ef6364f-7eed-4394-9de1-d6a924110730-kube-api-access-rb44l" (OuterVolumeSpecName: "kube-api-access-rb44l") pod "7ef6364f-7eed-4394-9de1-d6a924110730" (UID: "7ef6364f-7eed-4394-9de1-d6a924110730"). InnerVolumeSpecName "kube-api-access-rb44l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:11:34.528566 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.528533    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f215739-0873-43e7-bdf0-55dc38a599c7-kube-api-access-wg98c" (OuterVolumeSpecName: "kube-api-access-wg98c") pod "1f215739-0873-43e7-bdf0-55dc38a599c7" (UID: "1f215739-0873-43e7-bdf0-55dc38a599c7"). InnerVolumeSpecName "kube-api-access-wg98c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:11:34.627576 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.627523    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f215739-0873-43e7-bdf0-55dc38a599c7-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:11:34.627576 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.627544    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ef6364f-7eed-4394-9de1-d6a924110730-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:11:34.627576 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.627553    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg98c\" (UniqueName: \"kubernetes.io/projected/1f215739-0873-43e7-bdf0-55dc38a599c7-kube-api-access-wg98c\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:11:34.627576 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.627562    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f215739-0873-43e7-bdf0-55dc38a599c7-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:11:34.627576 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:34.627570    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb44l\" (UniqueName: \"kubernetes.io/projected/7ef6364f-7eed-4394-9de1-d6a924110730-kube-api-access-rb44l\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:11:35.099382 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.099352    2569 generic.go:356] "Generic (PLEG): container finished" podID="7ef6364f-7eed-4394-9de1-d6a924110730" containerID="f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684" exitCode=0
Jul 30 17:11:35.099739 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.099390    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"
Jul 30 17:11:35.099739 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.099440    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c" event={"ID":"7ef6364f-7eed-4394-9de1-d6a924110730","Type":"ContainerDied","Data":"f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684"}
Jul 30 17:11:35.099739 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.099486    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c" event={"ID":"7ef6364f-7eed-4394-9de1-d6a924110730","Type":"ContainerDied","Data":"2989cf667eeacd22f8d4d15f70f49c95fda65c5b63a83946edfba6ce2162bd0b"}
Jul 30 17:11:35.099739 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.099514    2569 scope.go:117] "RemoveContainer" containerID="f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684"
Jul 30 17:11:35.100895 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.100863    2569 generic.go:356] "Generic (PLEG): container finished" podID="de7ad458-04e4-4322-b234-aa69a419adf8" containerID="4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41" exitCode=0
Jul 30 17:11:35.101018 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.100926    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"
Jul 30 17:11:35.101018 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.100950    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z" event={"ID":"de7ad458-04e4-4322-b234-aa69a419adf8","Type":"ContainerDied","Data":"4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41"}
Jul 30 17:11:35.101018 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.100995    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z" event={"ID":"de7ad458-04e4-4322-b234-aa69a419adf8","Type":"ContainerDied","Data":"127f451acbc490797d91ca54582b5ac6a13a3b39002752ce8d416550f4b8c8b2"}
Jul 30 17:11:35.102389 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.102368    2569 generic.go:356] "Generic (PLEG): container finished" podID="1f215739-0873-43e7-bdf0-55dc38a599c7" containerID="7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f" exitCode=0
Jul 30 17:11:35.102486 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.102404    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"
Jul 30 17:11:35.102486 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.102406    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds" event={"ID":"1f215739-0873-43e7-bdf0-55dc38a599c7","Type":"ContainerDied","Data":"7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f"}
Jul 30 17:11:35.102583 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.102511    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds" event={"ID":"1f215739-0873-43e7-bdf0-55dc38a599c7","Type":"ContainerDied","Data":"150dc85a90c20eeeb751b17e1bf69bfbbb471709ac12e08a4cbec1467c2e2b8d"}
Jul 30 17:11:35.108378 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.108362    2569 scope.go:117] "RemoveContainer" containerID="f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684"
Jul 30 17:11:35.108658 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:11:35.108639    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684\": container with ID starting with f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684 not found: ID does not exist" containerID="f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684"
Jul 30 17:11:35.108725 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.108664    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684"} err="failed to get container status \"f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684\": rpc error: code = NotFound desc = could not find container \"f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684\": container with ID starting with f340e04fe3063e5c012486a7c1cf21119cf8edc1e24e5ab2f2605b2db2a1c684 not found: ID does not exist"
Jul 30 17:11:35.108725 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.108678    2569 scope.go:117] "RemoveContainer" containerID="4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41"
Jul 30 17:11:35.115495 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.115479    2569 scope.go:117] "RemoveContainer" containerID="4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41"
Jul 30 17:11:35.115760 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:11:35.115740    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41\": container with ID starting with 4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41 not found: ID does not exist" containerID="4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41"
Jul 30 17:11:35.115832 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.115763    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41"} err="failed to get container status \"4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41\": rpc error: code = NotFound desc = could not find container \"4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41\": container with ID starting with 4fa57664c695df8d46b04909c142a6fad6fe392ecc1d4a36e978f941f064fa41 not found: ID does not exist"
Jul 30 17:11:35.115832 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.115778    2569 scope.go:117] "RemoveContainer" containerID="7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f"
Jul 30 17:11:35.118720 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.118700    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"]
Jul 30 17:11:35.123359 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.123340    2569 scope.go:117] "RemoveContainer" containerID="7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f"
Jul 30 17:11:35.123591 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:11:35.123574    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f\": container with ID starting with 7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f not found: ID does not exist" containerID="7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f"
Jul 30 17:11:35.123668 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.123602    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f"} err="failed to get container status \"7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f\": rpc error: code = NotFound desc = could not find container \"7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f\": container with ID starting with 7846a9f9a81201155adf0a4179e841dbd62c4dfc745ec447367bb548c69c9c8f not found: ID does not exist"
Jul 30 17:11:35.126736 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.126717    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b68b3510-6f98cdb87c-tjw9z"]
Jul 30 17:11:35.142123 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.142101    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"]
Jul 30 17:11:35.143882 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.143859    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b68b3510-694f47bd9b-zwk5c"]
Jul 30 17:11:35.168699 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.168679    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"]
Jul 30 17:11:35.171216 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:35.171196    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b68b3510-75dd67d4c4-c46ds"]
Jul 30 17:11:36.999907 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:36.999876    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f215739-0873-43e7-bdf0-55dc38a599c7" path="/var/lib/kubelet/pods/1f215739-0873-43e7-bdf0-55dc38a599c7/volumes"
Jul 30 17:11:37.000263 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:37.000239    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ef6364f-7eed-4394-9de1-d6a924110730" path="/var/lib/kubelet/pods/7ef6364f-7eed-4394-9de1-d6a924110730/volumes"
Jul 30 17:11:37.000544 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:37.000531    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de7ad458-04e4-4322-b234-aa69a419adf8" path="/var/lib/kubelet/pods/de7ad458-04e4-4322-b234-aa69a419adf8/volumes"
Jul 30 17:11:38.627863 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.627831    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"]
Jul 30 17:11:38.628332 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.628312    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de7ad458-04e4-4322-b234-aa69a419adf8" containerName="payload-processing"
Jul 30 17:11:38.628409 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.628334    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="de7ad458-04e4-4322-b234-aa69a419adf8" containerName="payload-processing"
Jul 30 17:11:38.628409 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.628359    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef6364f-7eed-4394-9de1-d6a924110730" containerName="payload-pre-processing"
Jul 30 17:11:38.628409 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.628368    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef6364f-7eed-4394-9de1-d6a924110730" containerName="payload-pre-processing"
Jul 30 17:11:38.628409 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.628389    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f215739-0873-43e7-bdf0-55dc38a599c7" containerName="maas-api"
Jul 30 17:11:38.628409 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.628397    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f215739-0873-43e7-bdf0-55dc38a599c7" containerName="maas-api"
Jul 30 17:11:38.628684 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.628491    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f215739-0873-43e7-bdf0-55dc38a599c7" containerName="maas-api"
Jul 30 17:11:38.628684 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.628507    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef6364f-7eed-4394-9de1-d6a924110730" containerName="payload-pre-processing"
Jul 30 17:11:38.628684 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.628517    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="de7ad458-04e4-4322-b234-aa69a419adf8" containerName="payload-processing"
Jul 30 17:11:38.633055 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.633034    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:38.636013 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.635996    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b2e888e9-serving-cert\""
Jul 30 17:11:38.641267 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.641245    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"]
Jul 30 17:11:38.757561 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.757540    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-maas-api-tls\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:38.757716 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.757581    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgkvx\" (UniqueName: \"kubernetes.io/projected/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-kube-api-access-mgkvx\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:38.757716 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.757633    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-openshift-service-ca\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:38.858618 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.858580    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-openshift-service-ca\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:38.858715 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.858646    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-maas-api-tls\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:38.858715 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.858695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgkvx\" (UniqueName: \"kubernetes.io/projected/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-kube-api-access-mgkvx\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:38.858788 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:11:38.858729    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-serving-cert: secret "maas-api-e2e-ait-b2e888e9-serving-cert" not found
Jul 30 17:11:38.858819 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:11:38.858796    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-maas-api-tls podName:dc89c36b-39c9-4dce-bc31-a0705bf88fb9 nodeName:}" failed. No retries permitted until 2026-07-30 17:11:39.358779982 +0000 UTC m=+2126.961207590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-maas-api-tls") pod "maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq" (UID: "dc89c36b-39c9-4dce-bc31-a0705bf88fb9") : secret "maas-api-e2e-ait-b2e888e9-serving-cert" not found
Jul 30 17:11:38.859152 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.859133    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-openshift-service-ca\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:38.867508 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:38.867488    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgkvx\" (UniqueName: \"kubernetes.io/projected/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-kube-api-access-mgkvx\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:39.311654 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.311623    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"]
Jul 30 17:11:39.315069 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.315049    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:11:39.317700 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.317677    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b2e888e9\""
Jul 30 17:11:39.334994 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.334972    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"]
Jul 30 17:11:39.350973 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.350950    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"]
Jul 30 17:11:39.354105 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.354087    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:39.360774 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.360751    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b2e888e9-dockercfg-74t9b\""
Jul 30 17:11:39.361922 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.361901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-maas-api-tls\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:39.362001 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.361963    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce04f13a-13b3-47ae-9fde-c0da697002d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9\" (UID: \"ce04f13a-13b3-47ae-9fde-c0da697002d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:11:39.362068 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.362032    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5fqq\" (UniqueName: \"kubernetes.io/projected/ce04f13a-13b3-47ae-9fde-c0da697002d3-kube-api-access-j5fqq\") pod \"payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9\" (UID: \"ce04f13a-13b3-47ae-9fde-c0da697002d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:11:39.364161 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.364142    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-maas-api-tls\") pod \"maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:39.370506 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.370487    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"]
Jul 30 17:11:39.463237 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.463208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce04f13a-13b3-47ae-9fde-c0da697002d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9\" (UID: \"ce04f13a-13b3-47ae-9fde-c0da697002d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:11:39.463370 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.463242    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcxst\" (UniqueName: \"kubernetes.io/projected/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-kube-api-access-tcxst\") pod \"payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h\" (UID: \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\") " pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:39.463370 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.463266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5fqq\" (UniqueName: \"kubernetes.io/projected/ce04f13a-13b3-47ae-9fde-c0da697002d3-kube-api-access-j5fqq\") pod \"payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9\" (UID: \"ce04f13a-13b3-47ae-9fde-c0da697002d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:11:39.463370 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.463286    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-plugins-config-volume\") pod \"payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h\" (UID: \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\") " pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:39.463957 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.463934    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce04f13a-13b3-47ae-9fde-c0da697002d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9\" (UID: \"ce04f13a-13b3-47ae-9fde-c0da697002d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:11:39.472173 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.472144    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5fqq\" (UniqueName: \"kubernetes.io/projected/ce04f13a-13b3-47ae-9fde-c0da697002d3-kube-api-access-j5fqq\") pod \"payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9\" (UID: \"ce04f13a-13b3-47ae-9fde-c0da697002d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:11:39.543784 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.543762    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:39.564121 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.564061    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcxst\" (UniqueName: \"kubernetes.io/projected/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-kube-api-access-tcxst\") pod \"payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h\" (UID: \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\") " pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:39.564213 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.564124    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-plugins-config-volume\") pod \"payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h\" (UID: \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\") " pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:39.564850 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.564832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-plugins-config-volume\") pod \"payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h\" (UID: \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\") " pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:39.573468 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.573445    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcxst\" (UniqueName: \"kubernetes.io/projected/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-kube-api-access-tcxst\") pod \"payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h\" (UID: \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\") " pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:39.624817 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.624779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:11:39.663178 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.663140    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:39.681489 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.681030    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"]
Jul 30 17:11:39.773829 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.773739    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"]
Jul 30 17:11:39.779911 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:11:39.779878    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce04f13a_13b3_47ae_9fde_c0da697002d3.slice/crio-cb7e485ab3007eba685fc573ee57334741694787be090f7bd5b0cb9961cd0f03 WatchSource:0}: Error finding container cb7e485ab3007eba685fc573ee57334741694787be090f7bd5b0cb9961cd0f03: Status 404 returned error can't find the container with id cb7e485ab3007eba685fc573ee57334741694787be090f7bd5b0cb9961cd0f03
Jul 30 17:11:39.816518 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:39.816497    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"]
Jul 30 17:11:39.818473 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:11:39.818438    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b4a37d_bced_41b0_bde9_5aab7e584ff7.slice/crio-e87e3ec37ed87c4668e7b029cc6e96be09e58914c5c21be44cc5d30e8b074e6b WatchSource:0}: Error finding container e87e3ec37ed87c4668e7b029cc6e96be09e58914c5c21be44cc5d30e8b074e6b: Status 404 returned error can't find the container with id e87e3ec37ed87c4668e7b029cc6e96be09e58914c5c21be44cc5d30e8b074e6b
Jul 30 17:11:40.134507 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:40.134446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h" event={"ID":"a8b4a37d-bced-41b0-bde9-5aab7e584ff7","Type":"ContainerStarted","Data":"b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6"}
Jul 30 17:11:40.134663 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:40.134548    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:40.134663 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:40.134570    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h" event={"ID":"a8b4a37d-bced-41b0-bde9-5aab7e584ff7","Type":"ContainerStarted","Data":"e87e3ec37ed87c4668e7b029cc6e96be09e58914c5c21be44cc5d30e8b074e6b"}
Jul 30 17:11:40.135639 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:40.135595    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq" event={"ID":"dc89c36b-39c9-4dce-bc31-a0705bf88fb9","Type":"ContainerStarted","Data":"dee69d3a3d05f963a39d7a8f8cd9b9a0604bdf4ffdae77819b50a06f9dd92358"}
Jul 30 17:11:40.136844 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:40.136824    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9" event={"ID":"ce04f13a-13b3-47ae-9fde-c0da697002d3","Type":"ContainerStarted","Data":"d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8"}
Jul 30 17:11:40.136939 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:40.136856    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9" event={"ID":"ce04f13a-13b3-47ae-9fde-c0da697002d3","Type":"ContainerStarted","Data":"cb7e485ab3007eba685fc573ee57334741694787be090f7bd5b0cb9961cd0f03"}
Jul 30 17:11:40.136939 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:40.136875    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:11:40.153772 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:40.153736    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h" podStartSLOduration=1.153726044 podStartE2EDuration="1.153726044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:11:40.152159797 +0000 UTC m=+2127.754587436" watchObservedRunningTime="2026-07-30 17:11:40.153726044 +0000 UTC m=+2127.756153669"
Jul 30 17:11:40.177069 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:40.177028    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9" podStartSLOduration=1.177012478 podStartE2EDuration="1.177012478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:11:40.176190755 +0000 UTC m=+2127.778618385" watchObservedRunningTime="2026-07-30 17:11:40.177012478 +0000 UTC m=+2127.779440104"
Jul 30 17:11:41.144920 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:41.144825    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq" event={"ID":"dc89c36b-39c9-4dce-bc31-a0705bf88fb9","Type":"ContainerStarted","Data":"05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d"}
Jul 30 17:11:41.145356 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:41.145224    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:41.165375 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:41.165331    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq" podStartSLOduration=2.628352724 podStartE2EDuration="3.165315912s" totalImagesPullingTime="536.963188ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:11:41.163910455 +0000 UTC m=+2128.766338082" watchObservedRunningTime="2026-07-30 17:11:41.165315912 +0000 UTC m=+2128.767743538"
Jul 30 17:11:47.156243 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:47.156210    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:11:51.146492 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:51.146468    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:11:51.146917 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:11:51.146877    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:12:09.341114 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.341038    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"]
Jul 30 17:12:09.341584 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.341216    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq" podUID="dc89c36b-39c9-4dce-bc31-a0705bf88fb9" containerName="maas-api" containerID="cri-o://05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d" gracePeriod=30
Jul 30 17:12:09.408210 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.408180    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"]
Jul 30 17:12:09.408428 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.408404    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h" podUID="a8b4a37d-bced-41b0-bde9-5aab7e584ff7" containerName="payload-processing" containerID="cri-o://b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6" gracePeriod=30
Jul 30 17:12:09.416480 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.416437    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"]
Jul 30 17:12:09.416729 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.416682    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9" podUID="ce04f13a-13b3-47ae-9fde-c0da697002d3" containerName="payload-pre-processing" containerID="cri-o://d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8" gracePeriod=30
Jul 30 17:12:09.657301 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.657267    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:12:09.699860 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.699834    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:12:09.705687 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.705669    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:12:09.791656 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.791629    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcxst\" (UniqueName: \"kubernetes.io/projected/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-kube-api-access-tcxst\") pod \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\" (UID: \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\") "
Jul 30 17:12:09.791808 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.791673    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce04f13a-13b3-47ae-9fde-c0da697002d3-plugins-config-volume\") pod \"ce04f13a-13b3-47ae-9fde-c0da697002d3\" (UID: \"ce04f13a-13b3-47ae-9fde-c0da697002d3\") "
Jul 30 17:12:09.791808 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.791700    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-openshift-service-ca\") pod \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") "
Jul 30 17:12:09.791808 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.791715    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgkvx\" (UniqueName: \"kubernetes.io/projected/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-kube-api-access-mgkvx\") pod \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") "
Jul 30 17:12:09.791979 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.791810    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5fqq\" (UniqueName: \"kubernetes.io/projected/ce04f13a-13b3-47ae-9fde-c0da697002d3-kube-api-access-j5fqq\") pod \"ce04f13a-13b3-47ae-9fde-c0da697002d3\" (UID: \"ce04f13a-13b3-47ae-9fde-c0da697002d3\") "
Jul 30 17:12:09.791979 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.791903    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-plugins-config-volume\") pod \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\" (UID: \"a8b4a37d-bced-41b0-bde9-5aab7e584ff7\") "
Jul 30 17:12:09.791979 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.791970    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-maas-api-tls\") pod \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\" (UID: \"dc89c36b-39c9-4dce-bc31-a0705bf88fb9\") "
Jul 30 17:12:09.792135 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.792043    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dc89c36b-39c9-4dce-bc31-a0705bf88fb9" (UID: "dc89c36b-39c9-4dce-bc31-a0705bf88fb9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:12:09.792135 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.792065    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce04f13a-13b3-47ae-9fde-c0da697002d3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce04f13a-13b3-47ae-9fde-c0da697002d3" (UID: "ce04f13a-13b3-47ae-9fde-c0da697002d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:12:09.792308 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.792291    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce04f13a-13b3-47ae-9fde-c0da697002d3-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:09.792370 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.792311    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:09.792472 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.792449    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8b4a37d-bced-41b0-bde9-5aab7e584ff7" (UID: "a8b4a37d-bced-41b0-bde9-5aab7e584ff7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:12:09.793833 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.793810    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-kube-api-access-tcxst" (OuterVolumeSpecName: "kube-api-access-tcxst") pod "a8b4a37d-bced-41b0-bde9-5aab7e584ff7" (UID: "a8b4a37d-bced-41b0-bde9-5aab7e584ff7"). InnerVolumeSpecName "kube-api-access-tcxst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:12:09.794096 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.794069    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-kube-api-access-mgkvx" (OuterVolumeSpecName: "kube-api-access-mgkvx") pod "dc89c36b-39c9-4dce-bc31-a0705bf88fb9" (UID: "dc89c36b-39c9-4dce-bc31-a0705bf88fb9"). InnerVolumeSpecName "kube-api-access-mgkvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:12:09.794150 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.794122    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dc89c36b-39c9-4dce-bc31-a0705bf88fb9" (UID: "dc89c36b-39c9-4dce-bc31-a0705bf88fb9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:12:09.794347 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.794323    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce04f13a-13b3-47ae-9fde-c0da697002d3-kube-api-access-j5fqq" (OuterVolumeSpecName: "kube-api-access-j5fqq") pod "ce04f13a-13b3-47ae-9fde-c0da697002d3" (UID: "ce04f13a-13b3-47ae-9fde-c0da697002d3"). InnerVolumeSpecName "kube-api-access-j5fqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:12:09.892870 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.892849    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcxst\" (UniqueName: \"kubernetes.io/projected/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-kube-api-access-tcxst\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:09.892870 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.892869    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgkvx\" (UniqueName: \"kubernetes.io/projected/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-kube-api-access-mgkvx\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:09.892999 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.892878    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5fqq\" (UniqueName: \"kubernetes.io/projected/ce04f13a-13b3-47ae-9fde-c0da697002d3-kube-api-access-j5fqq\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:09.892999 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.892887    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8b4a37d-bced-41b0-bde9-5aab7e584ff7-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:09.892999 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:09.892896    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc89c36b-39c9-4dce-bc31-a0705bf88fb9-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:10.273220 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.273151    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8b4a37d-bced-41b0-bde9-5aab7e584ff7" containerID="b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6" exitCode=0
Jul 30 17:12:10.273220 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.273188    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"
Jul 30 17:12:10.273220 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.273218    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h" event={"ID":"a8b4a37d-bced-41b0-bde9-5aab7e584ff7","Type":"ContainerDied","Data":"b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6"}
Jul 30 17:12:10.273463 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.273256    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h" event={"ID":"a8b4a37d-bced-41b0-bde9-5aab7e584ff7","Type":"ContainerDied","Data":"e87e3ec37ed87c4668e7b029cc6e96be09e58914c5c21be44cc5d30e8b074e6b"}
Jul 30 17:12:10.273463 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.273276    2569 scope.go:117] "RemoveContainer" containerID="b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6"
Jul 30 17:12:10.274546 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.274526    2569 generic.go:356] "Generic (PLEG): container finished" podID="dc89c36b-39c9-4dce-bc31-a0705bf88fb9" containerID="05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d" exitCode=0
Jul 30 17:12:10.274647 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.274566    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"
Jul 30 17:12:10.274647 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.274627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq" event={"ID":"dc89c36b-39c9-4dce-bc31-a0705bf88fb9","Type":"ContainerDied","Data":"05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d"}
Jul 30 17:12:10.274777 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.274669    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq" event={"ID":"dc89c36b-39c9-4dce-bc31-a0705bf88fb9","Type":"ContainerDied","Data":"dee69d3a3d05f963a39d7a8f8cd9b9a0604bdf4ffdae77819b50a06f9dd92358"}
Jul 30 17:12:10.276072 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.276048    2569 generic.go:356] "Generic (PLEG): container finished" podID="ce04f13a-13b3-47ae-9fde-c0da697002d3" containerID="d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8" exitCode=0
Jul 30 17:12:10.276072 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.276077    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"
Jul 30 17:12:10.276225 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.276125    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9" event={"ID":"ce04f13a-13b3-47ae-9fde-c0da697002d3","Type":"ContainerDied","Data":"d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8"}
Jul 30 17:12:10.276225 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.276173    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9" event={"ID":"ce04f13a-13b3-47ae-9fde-c0da697002d3","Type":"ContainerDied","Data":"cb7e485ab3007eba685fc573ee57334741694787be090f7bd5b0cb9961cd0f03"}
Jul 30 17:12:10.283319 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.283297    2569 scope.go:117] "RemoveContainer" containerID="b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6"
Jul 30 17:12:10.283594 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:12:10.283575    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6\": container with ID starting with b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6 not found: ID does not exist" containerID="b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6"
Jul 30 17:12:10.283750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.283617    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6"} err="failed to get container status \"b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6\": rpc error: code = NotFound desc = could not find container \"b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6\": container with ID starting with b0b04e89de9ec4f73bb9e2dbfd59ca16ac7bd1beffd8bef93d58106d999675d6 not found: ID does not exist"
Jul 30 17:12:10.283750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.283639    2569 scope.go:117] "RemoveContainer" containerID="05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d"
Jul 30 17:12:10.291319 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.291299    2569 scope.go:117] "RemoveContainer" containerID="05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d"
Jul 30 17:12:10.291558 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:12:10.291542    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d\": container with ID starting with 05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d not found: ID does not exist" containerID="05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d"
Jul 30 17:12:10.291615 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.291563    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d"} err="failed to get container status \"05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d\": rpc error: code = NotFound desc = could not find container \"05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d\": container with ID starting with 05963017a70ec837f139be7d03f71c08c2c0f5c781020b894f0ba44f9580594d not found: ID does not exist"
Jul 30 17:12:10.291615 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.291578    2569 scope.go:117] "RemoveContainer" containerID="d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8"
Jul 30 17:12:10.298333 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.298317    2569 scope.go:117] "RemoveContainer" containerID="d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8"
Jul 30 17:12:10.298550 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:12:10.298535    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8\": container with ID starting with d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8 not found: ID does not exist" containerID="d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8"
Jul 30 17:12:10.298590 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.298555    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8"} err="failed to get container status \"d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8\": rpc error: code = NotFound desc = could not find container \"d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8\": container with ID starting with d0bf83983a8d342026b5382d12cd7e57ca4c45ad4608e26180bf4b27d38f08c8 not found: ID does not exist"
Jul 30 17:12:10.305149 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.305124    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"]
Jul 30 17:12:10.311935 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.311913    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b2e888e9-866b7b5cb9-2hrrq"]
Jul 30 17:12:10.324474 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.324453    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"]
Jul 30 17:12:10.331114 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.331092    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b2e888e9-6fdbc9b84d-bjfw9"]
Jul 30 17:12:10.343635 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.343598    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"]
Jul 30 17:12:10.352972 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:10.352952    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b2e888e9-6fb4b886d6-grb5h"]
Jul 30 17:12:11.001091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:11.001053    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8b4a37d-bced-41b0-bde9-5aab7e584ff7" path="/var/lib/kubelet/pods/a8b4a37d-bced-41b0-bde9-5aab7e584ff7/volumes"
Jul 30 17:12:11.001400 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:11.001385    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce04f13a-13b3-47ae-9fde-c0da697002d3" path="/var/lib/kubelet/pods/ce04f13a-13b3-47ae-9fde-c0da697002d3/volumes"
Jul 30 17:12:11.001744 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:11.001730    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc89c36b-39c9-4dce-bc31-a0705bf88fb9" path="/var/lib/kubelet/pods/dc89c36b-39c9-4dce-bc31-a0705bf88fb9/volumes"
Jul 30 17:12:15.046353 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046317    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"]
Jul 30 17:12:15.046750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046690    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc89c36b-39c9-4dce-bc31-a0705bf88fb9" containerName="maas-api"
Jul 30 17:12:15.046750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046702    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc89c36b-39c9-4dce-bc31-a0705bf88fb9" containerName="maas-api"
Jul 30 17:12:15.046750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046722    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8b4a37d-bced-41b0-bde9-5aab7e584ff7" containerName="payload-processing"
Jul 30 17:12:15.046750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046728    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8b4a37d-bced-41b0-bde9-5aab7e584ff7" containerName="payload-processing"
Jul 30 17:12:15.046750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046740    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce04f13a-13b3-47ae-9fde-c0da697002d3" containerName="payload-pre-processing"
Jul 30 17:12:15.046750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046746    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce04f13a-13b3-47ae-9fde-c0da697002d3" containerName="payload-pre-processing"
Jul 30 17:12:15.046933 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046800    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc89c36b-39c9-4dce-bc31-a0705bf88fb9" containerName="maas-api"
Jul 30 17:12:15.046933 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046808    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce04f13a-13b3-47ae-9fde-c0da697002d3" containerName="payload-pre-processing"
Jul 30 17:12:15.046933 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.046813    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8b4a37d-bced-41b0-bde9-5aab7e584ff7" containerName="payload-processing"
Jul 30 17:12:15.051589 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.051573    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.054671 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.054653    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-4548eef0-serving-cert\""
Jul 30 17:12:15.069055 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.069030    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"]
Jul 30 17:12:15.139134 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.139096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbrkw\" (UniqueName: \"kubernetes.io/projected/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-kube-api-access-kbrkw\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.139317 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.139162    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-maas-api-tls\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.139317 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.139203    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-openshift-service-ca\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.240128 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.240099    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-openshift-service-ca\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.240291 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.240161    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbrkw\" (UniqueName: \"kubernetes.io/projected/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-kube-api-access-kbrkw\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.240291 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.240202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-maas-api-tls\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.240403 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:12:15.240296    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-serving-cert: secret "maas-api-e2e-derive-4548eef0-serving-cert" not found
Jul 30 17:12:15.240403 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:12:15.240349    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-maas-api-tls podName:7d92f1b2-cb89-4185-bde6-7c3bb708e6b2 nodeName:}" failed. No retries permitted until 2026-07-30 17:12:15.740335022 +0000 UTC m=+2163.342762626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-maas-api-tls") pod "maas-api-e2e-derive-4548eef0-79c5d59688-47c9k" (UID: "7d92f1b2-cb89-4185-bde6-7c3bb708e6b2") : secret "maas-api-e2e-derive-4548eef0-serving-cert" not found
Jul 30 17:12:15.240778 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.240758    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-openshift-service-ca\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.261078 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.261052    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbrkw\" (UniqueName: \"kubernetes.io/projected/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-kube-api-access-kbrkw\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.744667 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.744638    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-maas-api-tls\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.746957 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.746925    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-maas-api-tls\") pod \"maas-api-e2e-derive-4548eef0-79c5d59688-47c9k\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:15.897170 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.897141    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"]
Jul 30 17:12:15.905472 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.905448    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:15.908228 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.908210    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-4548eef0-dockercfg-d8fm9\""
Jul 30 17:12:15.908676 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.908662    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-4548eef0\""
Jul 30 17:12:15.917374 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.917355    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"]
Jul 30 17:12:15.961727 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:15.961704    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:16.009552 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.009487    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"]
Jul 30 17:12:16.017623 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.014460    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:16.036488 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.036409    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"]
Jul 30 17:12:16.048206 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.048164    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2\" (UID: \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:16.048890 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.048857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfhhm\" (UniqueName: \"kubernetes.io/projected/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-kube-api-access-jfhhm\") pod \"payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2\" (UID: \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:16.133419 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.133388    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"]
Jul 30 17:12:16.137804 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:12:16.137773    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d92f1b2_cb89_4185_bde6_7c3bb708e6b2.slice/crio-e4132c14023257eeeb80d51402bca36af16b7a90b423260754f37c04771e73c6 WatchSource:0}: Error finding container e4132c14023257eeeb80d51402bca36af16b7a90b423260754f37c04771e73c6: Status 404 returned error can't find the container with id e4132c14023257eeeb80d51402bca36af16b7a90b423260754f37c04771e73c6
Jul 30 17:12:16.149899 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.149876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88da1e0-6f25-48be-a72c-b3964f68aaa2-plugins-config-volume\") pod \"payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg\" (UID: \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\") " pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:16.150001 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.149917    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnwc5\" (UniqueName: \"kubernetes.io/projected/a88da1e0-6f25-48be-a72c-b3964f68aaa2-kube-api-access-gnwc5\") pod \"payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg\" (UID: \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\") " pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:16.150079 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.150000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2\" (UID: \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:16.150132 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.150114    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfhhm\" (UniqueName: \"kubernetes.io/projected/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-kube-api-access-jfhhm\") pod \"payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2\" (UID: \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:16.150902 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.150884    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2\" (UID: \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:16.159352 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.159335    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfhhm\" (UniqueName: \"kubernetes.io/projected/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-kube-api-access-jfhhm\") pod \"payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2\" (UID: \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:16.216278 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.216255    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:16.251297 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.251264    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88da1e0-6f25-48be-a72c-b3964f68aaa2-plugins-config-volume\") pod \"payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg\" (UID: \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\") " pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:16.251420 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.251302    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnwc5\" (UniqueName: \"kubernetes.io/projected/a88da1e0-6f25-48be-a72c-b3964f68aaa2-kube-api-access-gnwc5\") pod \"payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg\" (UID: \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\") " pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:16.252208 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.252185    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88da1e0-6f25-48be-a72c-b3964f68aaa2-plugins-config-volume\") pod \"payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg\" (UID: \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\") " pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:16.260388 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.260333    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnwc5\" (UniqueName: \"kubernetes.io/projected/a88da1e0-6f25-48be-a72c-b3964f68aaa2-kube-api-access-gnwc5\") pod \"payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg\" (UID: \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\") " pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:16.308040 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.308006    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k" event={"ID":"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2","Type":"ContainerStarted","Data":"e4132c14023257eeeb80d51402bca36af16b7a90b423260754f37c04771e73c6"}
Jul 30 17:12:16.335259 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.335233    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:16.348086 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.348046    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"]
Jul 30 17:12:16.349932 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:12:16.349904    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44c740f_ebe0_4f19_a80a_c27fc1bc274a.slice/crio-21807535169f78b2c72198a251a9cc46341b29c916e81fc120e177980c27018f WatchSource:0}: Error finding container 21807535169f78b2c72198a251a9cc46341b29c916e81fc120e177980c27018f: Status 404 returned error can't find the container with id 21807535169f78b2c72198a251a9cc46341b29c916e81fc120e177980c27018f
Jul 30 17:12:16.474986 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:16.474963    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"]
Jul 30 17:12:16.476942 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:12:16.476918    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88da1e0_6f25_48be_a72c_b3964f68aaa2.slice/crio-6532899e8930fb92fb9bc8c98c9b1fa99123bc909a41c300c126e945f8382b75 WatchSource:0}: Error finding container 6532899e8930fb92fb9bc8c98c9b1fa99123bc909a41c300c126e945f8382b75: Status 404 returned error can't find the container with id 6532899e8930fb92fb9bc8c98c9b1fa99123bc909a41c300c126e945f8382b75
Jul 30 17:12:17.313901 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.313868    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k" event={"ID":"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2","Type":"ContainerStarted","Data":"0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653"}
Jul 30 17:12:17.314276 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.313967    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:17.315357 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.315333    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2" event={"ID":"d44c740f-ebe0-4f19-a80a-c27fc1bc274a","Type":"ContainerStarted","Data":"ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9"}
Jul 30 17:12:17.315488 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.315370    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2" event={"ID":"d44c740f-ebe0-4f19-a80a-c27fc1bc274a","Type":"ContainerStarted","Data":"21807535169f78b2c72198a251a9cc46341b29c916e81fc120e177980c27018f"}
Jul 30 17:12:17.315488 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.315480    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:17.316799 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.316778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg" event={"ID":"a88da1e0-6f25-48be-a72c-b3964f68aaa2","Type":"ContainerStarted","Data":"7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c"}
Jul 30 17:12:17.317052 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.316810    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg" event={"ID":"a88da1e0-6f25-48be-a72c-b3964f68aaa2","Type":"ContainerStarted","Data":"6532899e8930fb92fb9bc8c98c9b1fa99123bc909a41c300c126e945f8382b75"}
Jul 30 17:12:17.317052 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.316829    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:17.333861 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.333823    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k" podStartSLOduration=1.8412576139999999 podStartE2EDuration="2.333811164s" totalImagesPullingTime="492.55355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:12:17.331597623 +0000 UTC m=+2164.934025249" watchObservedRunningTime="2026-07-30 17:12:17.333811164 +0000 UTC m=+2164.936238790"
Jul 30 17:12:17.353467 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.353423    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg" podStartSLOduration=2.35340782 podStartE2EDuration="2.35340782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:12:17.352965396 +0000 UTC m=+2164.955393026" watchObservedRunningTime="2026-07-30 17:12:17.35340782 +0000 UTC m=+2164.955835446"
Jul 30 17:12:17.372715 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:17.372675    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2" podStartSLOduration=2.372663723 podStartE2EDuration="2.372663723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:12:17.371267666 +0000 UTC m=+2164.973695290" watchObservedRunningTime="2026-07-30 17:12:17.372663723 +0000 UTC m=+2164.975091348"
Jul 30 17:12:23.327275 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:23.327249    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:28.324661 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:28.324634    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:28.325101 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:28.324832    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:45.646235 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.646199    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"]
Jul 30 17:12:45.646846 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.646481    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k" podUID="7d92f1b2-cb89-4185-bde6-7c3bb708e6b2" containerName="maas-api" containerID="cri-o://0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653" gracePeriod=30
Jul 30 17:12:45.764312 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.764281    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"]
Jul 30 17:12:45.764574 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.764527    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg" podUID="a88da1e0-6f25-48be-a72c-b3964f68aaa2" containerName="payload-processing" containerID="cri-o://7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c" gracePeriod=30
Jul 30 17:12:45.773885 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.773813    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"]
Jul 30 17:12:45.774261 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.774110    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2" podUID="d44c740f-ebe0-4f19-a80a-c27fc1bc274a" containerName="payload-pre-processing" containerID="cri-o://ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9" gracePeriod=30
Jul 30 17:12:45.914648 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.914596    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:45.985038 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.985012    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-openshift-service-ca\") pod \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") "
Jul 30 17:12:45.985161 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.985121    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbrkw\" (UniqueName: \"kubernetes.io/projected/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-kube-api-access-kbrkw\") pod \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") "
Jul 30 17:12:45.985161 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.985155    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-maas-api-tls\") pod \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\" (UID: \"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2\") "
Jul 30 17:12:45.985699 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.985589    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d92f1b2-cb89-4185-bde6-7c3bb708e6b2" (UID: "7d92f1b2-cb89-4185-bde6-7c3bb708e6b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:12:45.987937 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.987908    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d92f1b2-cb89-4185-bde6-7c3bb708e6b2" (UID: "7d92f1b2-cb89-4185-bde6-7c3bb708e6b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:12:45.987937 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:45.987907    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-kube-api-access-kbrkw" (OuterVolumeSpecName: "kube-api-access-kbrkw") pod "7d92f1b2-cb89-4185-bde6-7c3bb708e6b2" (UID: "7d92f1b2-cb89-4185-bde6-7c3bb708e6b2"). InnerVolumeSpecName "kube-api-access-kbrkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:12:46.033355 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.033331    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:46.040713 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.040694    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:46.086326 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.086212    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88da1e0-6f25-48be-a72c-b3964f68aaa2-plugins-config-volume\") pod \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\" (UID: \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\") "
Jul 30 17:12:46.086326 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.086297    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfhhm\" (UniqueName: \"kubernetes.io/projected/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-kube-api-access-jfhhm\") pod \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\" (UID: \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\") "
Jul 30 17:12:46.086533 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.086377    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnwc5\" (UniqueName: \"kubernetes.io/projected/a88da1e0-6f25-48be-a72c-b3964f68aaa2-kube-api-access-gnwc5\") pod \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\" (UID: \"a88da1e0-6f25-48be-a72c-b3964f68aaa2\") "
Jul 30 17:12:46.086533 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.086411    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-plugins-config-volume\") pod \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\" (UID: \"d44c740f-ebe0-4f19-a80a-c27fc1bc274a\") "
Jul 30 17:12:46.087095 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.086970    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a88da1e0-6f25-48be-a72c-b3964f68aaa2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a88da1e0-6f25-48be-a72c-b3964f68aaa2" (UID: "a88da1e0-6f25-48be-a72c-b3964f68aaa2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:12:46.088422 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.087343    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbrkw\" (UniqueName: \"kubernetes.io/projected/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-kube-api-access-kbrkw\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:46.088422 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.087370    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:46.088578 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.088442    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d44c740f-ebe0-4f19-a80a-c27fc1bc274a" (UID: "d44c740f-ebe0-4f19-a80a-c27fc1bc274a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:12:46.088694 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.087384    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a88da1e0-6f25-48be-a72c-b3964f68aaa2-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:46.088814 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.088705    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:46.089760 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.089737    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-kube-api-access-jfhhm" (OuterVolumeSpecName: "kube-api-access-jfhhm") pod "d44c740f-ebe0-4f19-a80a-c27fc1bc274a" (UID: "d44c740f-ebe0-4f19-a80a-c27fc1bc274a"). InnerVolumeSpecName "kube-api-access-jfhhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:12:46.090449 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.090418    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a88da1e0-6f25-48be-a72c-b3964f68aaa2-kube-api-access-gnwc5" (OuterVolumeSpecName: "kube-api-access-gnwc5") pod "a88da1e0-6f25-48be-a72c-b3964f68aaa2" (UID: "a88da1e0-6f25-48be-a72c-b3964f68aaa2"). InnerVolumeSpecName "kube-api-access-gnwc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:12:46.189456 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.189390    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnwc5\" (UniqueName: \"kubernetes.io/projected/a88da1e0-6f25-48be-a72c-b3964f68aaa2-kube-api-access-gnwc5\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:46.189456 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.189414    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:46.189456 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.189423    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfhhm\" (UniqueName: \"kubernetes.io/projected/d44c740f-ebe0-4f19-a80a-c27fc1bc274a-kube-api-access-jfhhm\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:12:46.447519 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.447422    2569 generic.go:356] "Generic (PLEG): container finished" podID="7d92f1b2-cb89-4185-bde6-7c3bb708e6b2" containerID="0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653" exitCode=0
Jul 30 17:12:46.447709 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.447648    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"
Jul 30 17:12:46.447868 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.447840    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k" event={"ID":"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2","Type":"ContainerDied","Data":"0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653"}
Jul 30 17:12:46.447984 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.447967    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k" event={"ID":"7d92f1b2-cb89-4185-bde6-7c3bb708e6b2","Type":"ContainerDied","Data":"e4132c14023257eeeb80d51402bca36af16b7a90b423260754f37c04771e73c6"}
Jul 30 17:12:46.448054 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.448033    2569 scope.go:117] "RemoveContainer" containerID="0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653"
Jul 30 17:12:46.449245 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.449220    2569 generic.go:356] "Generic (PLEG): container finished" podID="d44c740f-ebe0-4f19-a80a-c27fc1bc274a" containerID="ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9" exitCode=0
Jul 30 17:12:46.449376 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.449283    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"
Jul 30 17:12:46.449376 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.449316    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2" event={"ID":"d44c740f-ebe0-4f19-a80a-c27fc1bc274a","Type":"ContainerDied","Data":"ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9"}
Jul 30 17:12:46.449376 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.449355    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2" event={"ID":"d44c740f-ebe0-4f19-a80a-c27fc1bc274a","Type":"ContainerDied","Data":"21807535169f78b2c72198a251a9cc46341b29c916e81fc120e177980c27018f"}
Jul 30 17:12:46.450776 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.450754    2569 generic.go:356] "Generic (PLEG): container finished" podID="a88da1e0-6f25-48be-a72c-b3964f68aaa2" containerID="7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c" exitCode=0
Jul 30 17:12:46.450883 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.450816    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg" event={"ID":"a88da1e0-6f25-48be-a72c-b3964f68aaa2","Type":"ContainerDied","Data":"7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c"}
Jul 30 17:12:46.450883 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.450851    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg" event={"ID":"a88da1e0-6f25-48be-a72c-b3964f68aaa2","Type":"ContainerDied","Data":"6532899e8930fb92fb9bc8c98c9b1fa99123bc909a41c300c126e945f8382b75"}
Jul 30 17:12:46.450883 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.450855    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"
Jul 30 17:12:46.457150 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.457134    2569 scope.go:117] "RemoveContainer" containerID="0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653"
Jul 30 17:12:46.457421 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:12:46.457378    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653\": container with ID starting with 0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653 not found: ID does not exist" containerID="0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653"
Jul 30 17:12:46.457515 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.457414    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653"} err="failed to get container status \"0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653\": rpc error: code = NotFound desc = could not find container \"0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653\": container with ID starting with 0fa3f4a29d41e22e07edc6f82678ea8769ecb8da8ff14ed63e4c9b73335be653 not found: ID does not exist"
Jul 30 17:12:46.457515 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.457435    2569 scope.go:117] "RemoveContainer" containerID="ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9"
Jul 30 17:12:46.465521 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.465507    2569 scope.go:117] "RemoveContainer" containerID="ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9"
Jul 30 17:12:46.465783 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:12:46.465766    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9\": container with ID starting with ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9 not found: ID does not exist" containerID="ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9"
Jul 30 17:12:46.465839 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.465789    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9"} err="failed to get container status \"ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9\": rpc error: code = NotFound desc = could not find container \"ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9\": container with ID starting with ffe0dcf90cd1050089c51cbbdc33589b8ba7a38bde2351a17026933444c8b5e9 not found: ID does not exist"
Jul 30 17:12:46.465839 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.465804    2569 scope.go:117] "RemoveContainer" containerID="7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c"
Jul 30 17:12:46.472675 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.472657    2569 scope.go:117] "RemoveContainer" containerID="7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c"
Jul 30 17:12:46.472912 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:12:46.472896    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c\": container with ID starting with 7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c not found: ID does not exist" containerID="7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c"
Jul 30 17:12:46.472957 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.472918    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c"} err="failed to get container status \"7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c\": rpc error: code = NotFound desc = could not find container \"7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c\": container with ID starting with 7aa52ed882e2bbd38d778c69e6e5bd21ca8191e8397144ce491254b08603b49c not found: ID does not exist"
Jul 30 17:12:46.476288 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.476269    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"]
Jul 30 17:12:46.482360 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.482341    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4548eef0-79c5d59688-47c9k"]
Jul 30 17:12:46.492284 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.492265    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"]
Jul 30 17:12:46.496380 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.496358    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4548eef0-d6d5b5f96-ppmb2"]
Jul 30 17:12:46.506899 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.506882    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"]
Jul 30 17:12:46.510874 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:46.510855    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4548eef0-c89dbfd78-p9msg"]
Jul 30 17:12:47.000598 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:47.000570    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d92f1b2-cb89-4185-bde6-7c3bb708e6b2" path="/var/lib/kubelet/pods/7d92f1b2-cb89-4185-bde6-7c3bb708e6b2/volumes"
Jul 30 17:12:47.000973 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:47.000957    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a88da1e0-6f25-48be-a72c-b3964f68aaa2" path="/var/lib/kubelet/pods/a88da1e0-6f25-48be-a72c-b3964f68aaa2/volumes"
Jul 30 17:12:47.001312 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:12:47.001292    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d44c740f-ebe0-4f19-a80a-c27fc1bc274a" path="/var/lib/kubelet/pods/d44c740f-ebe0-4f19-a80a-c27fc1bc274a/volumes"
Jul 30 17:15:00.404576 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.404539    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"]
Jul 30 17:15:00.405064 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.405036    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d44c740f-ebe0-4f19-a80a-c27fc1bc274a" containerName="payload-pre-processing"
Jul 30 17:15:00.405064 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.405052    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d44c740f-ebe0-4f19-a80a-c27fc1bc274a" containerName="payload-pre-processing"
Jul 30 17:15:00.405137 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.405070    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d92f1b2-cb89-4185-bde6-7c3bb708e6b2" containerName="maas-api"
Jul 30 17:15:00.405137 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.405079    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d92f1b2-cb89-4185-bde6-7c3bb708e6b2" containerName="maas-api"
Jul 30 17:15:00.405137 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.405097    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a88da1e0-6f25-48be-a72c-b3964f68aaa2" containerName="payload-processing"
Jul 30 17:15:00.405137 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.405105    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a88da1e0-6f25-48be-a72c-b3964f68aaa2" containerName="payload-processing"
Jul 30 17:15:00.405253 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.405184    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d44c740f-ebe0-4f19-a80a-c27fc1bc274a" containerName="payload-pre-processing"
Jul 30 17:15:00.405253 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.405196    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a88da1e0-6f25-48be-a72c-b3964f68aaa2" containerName="payload-processing"
Jul 30 17:15:00.405253 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.405205    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d92f1b2-cb89-4185-bde6-7c3bb708e6b2" containerName="maas-api"
Jul 30 17:15:00.408457 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.408438    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:00.411028 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.410999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39224a0a-5fd7-4e7e-b80a-e3a501be0550-openshift-service-ca\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:00.411165 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.411086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39224a0a-5fd7-4e7e-b80a-e3a501be0550-maas-api-tls\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:00.411165 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.411157    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e6f2963a-serving-cert\""
Jul 30 17:15:00.411353 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.411321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdf9c\" (UniqueName: \"kubernetes.io/projected/39224a0a-5fd7-4e7e-b80a-e3a501be0550-kube-api-access-sdf9c\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:00.417751 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.417729    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"]
Jul 30 17:15:00.511712 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.511688    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39224a0a-5fd7-4e7e-b80a-e3a501be0550-openshift-service-ca\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:00.511817 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.511722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39224a0a-5fd7-4e7e-b80a-e3a501be0550-maas-api-tls\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:00.511817 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.511773    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdf9c\" (UniqueName: \"kubernetes.io/projected/39224a0a-5fd7-4e7e-b80a-e3a501be0550-kube-api-access-sdf9c\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:00.511915 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:15:00.511892    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-serving-cert: secret "maas-api-e2e-mt-e6f2963a-serving-cert" not found
Jul 30 17:15:00.512018 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:15:00.511963    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39224a0a-5fd7-4e7e-b80a-e3a501be0550-maas-api-tls podName:39224a0a-5fd7-4e7e-b80a-e3a501be0550 nodeName:}" failed. No retries permitted until 2026-07-30 17:15:01.011944815 +0000 UTC m=+2328.614372425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/39224a0a-5fd7-4e7e-b80a-e3a501be0550-maas-api-tls") pod "maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v" (UID: "39224a0a-5fd7-4e7e-b80a-e3a501be0550") : secret "maas-api-e2e-mt-e6f2963a-serving-cert" not found
Jul 30 17:15:00.512360 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.512341    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39224a0a-5fd7-4e7e-b80a-e3a501be0550-openshift-service-ca\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:00.521169 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:00.521148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdf9c\" (UniqueName: \"kubernetes.io/projected/39224a0a-5fd7-4e7e-b80a-e3a501be0550-kube-api-access-sdf9c\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:01.015297 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.015268    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39224a0a-5fd7-4e7e-b80a-e3a501be0550-maas-api-tls\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:01.017978 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.017950    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39224a0a-5fd7-4e7e-b80a-e3a501be0550-maas-api-tls\") pod \"maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:01.019713 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.019677    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:01.158588 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.158563    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"]
Jul 30 17:15:01.159673 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:15:01.159629    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39224a0a_5fd7_4e7e_b80a_e3a501be0550.slice/crio-4536d216762d2da3498ab55216429c778866db2b2752222554b00a2d0332639b WatchSource:0}: Error finding container 4536d216762d2da3498ab55216429c778866db2b2752222554b00a2d0332639b: Status 404 returned error can't find the container with id 4536d216762d2da3498ab55216429c778866db2b2752222554b00a2d0332639b
Jul 30 17:15:01.236108 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.236084    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"]
Jul 30 17:15:01.240237 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.240219    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:01.242656 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.242632    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"]
Jul 30 17:15:01.242935 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.242913    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e6f2963a\""
Jul 30 17:15:01.245731 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.245704    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:01.248548 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.248527    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e6f2963a-dockercfg-ckxvp\""
Jul 30 17:15:01.254716 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.254695    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"]
Jul 30 17:15:01.259828 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.259809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"]
Jul 30 17:15:01.318373 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.318313    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7ca78b-3eff-45dc-972f-65433935ad77-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2\" (UID: \"ac7ca78b-3eff-45dc-972f-65433935ad77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:01.318500 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.318377    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94wfv\" (UniqueName: \"kubernetes.io/projected/ac7ca78b-3eff-45dc-972f-65433935ad77-kube-api-access-94wfv\") pod \"payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2\" (UID: \"ac7ca78b-3eff-45dc-972f-65433935ad77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:01.419201 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.419177    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed33416-81ef-4ee0-9050-fec3160120fe-plugins-config-volume\") pod \"payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr\" (UID: \"eed33416-81ef-4ee0-9050-fec3160120fe\") " pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:01.419500 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.419205    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmhdn\" (UniqueName: \"kubernetes.io/projected/eed33416-81ef-4ee0-9050-fec3160120fe-kube-api-access-rmhdn\") pod \"payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr\" (UID: \"eed33416-81ef-4ee0-9050-fec3160120fe\") " pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:01.419500 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.419254    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94wfv\" (UniqueName: \"kubernetes.io/projected/ac7ca78b-3eff-45dc-972f-65433935ad77-kube-api-access-94wfv\") pod \"payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2\" (UID: \"ac7ca78b-3eff-45dc-972f-65433935ad77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:01.419500 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.419304    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7ca78b-3eff-45dc-972f-65433935ad77-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2\" (UID: \"ac7ca78b-3eff-45dc-972f-65433935ad77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:01.419954 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.419937    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7ca78b-3eff-45dc-972f-65433935ad77-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2\" (UID: \"ac7ca78b-3eff-45dc-972f-65433935ad77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:01.437385 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.437362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94wfv\" (UniqueName: \"kubernetes.io/projected/ac7ca78b-3eff-45dc-972f-65433935ad77-kube-api-access-94wfv\") pod \"payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2\" (UID: \"ac7ca78b-3eff-45dc-972f-65433935ad77\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:01.520098 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.520075    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed33416-81ef-4ee0-9050-fec3160120fe-plugins-config-volume\") pod \"payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr\" (UID: \"eed33416-81ef-4ee0-9050-fec3160120fe\") " pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:01.520210 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.520103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmhdn\" (UniqueName: \"kubernetes.io/projected/eed33416-81ef-4ee0-9050-fec3160120fe-kube-api-access-rmhdn\") pod \"payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr\" (UID: \"eed33416-81ef-4ee0-9050-fec3160120fe\") " pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:01.520822 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.520803    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed33416-81ef-4ee0-9050-fec3160120fe-plugins-config-volume\") pod \"payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr\" (UID: \"eed33416-81ef-4ee0-9050-fec3160120fe\") " pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:01.543931 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.543908    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmhdn\" (UniqueName: \"kubernetes.io/projected/eed33416-81ef-4ee0-9050-fec3160120fe-kube-api-access-rmhdn\") pod \"payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr\" (UID: \"eed33416-81ef-4ee0-9050-fec3160120fe\") " pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:01.550634 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.550577    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:01.559372 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.559353    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:01.702023 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.701997    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"]
Jul 30 17:15:01.705042 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:15:01.705011    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed33416_81ef_4ee0_9050_fec3160120fe.slice/crio-9b4549a66ee86727849976802c5e3d322355603709161a3c141119e2b06e29f1 WatchSource:0}: Error finding container 9b4549a66ee86727849976802c5e3d322355603709161a3c141119e2b06e29f1: Status 404 returned error can't find the container with id 9b4549a66ee86727849976802c5e3d322355603709161a3c141119e2b06e29f1
Jul 30 17:15:01.931021 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:01.930635    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"]
Jul 30 17:15:02.063858 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.063826    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2" event={"ID":"ac7ca78b-3eff-45dc-972f-65433935ad77","Type":"ContainerStarted","Data":"114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084"}
Jul 30 17:15:02.063972 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.063874    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2" event={"ID":"ac7ca78b-3eff-45dc-972f-65433935ad77","Type":"ContainerStarted","Data":"4b3df2f3fe6987e21bcabcb90408fce05fc74c3f9e173b1e933b9ed4ec2683e0"}
Jul 30 17:15:02.063972 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.063942    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:02.065430 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.065402    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr" event={"ID":"eed33416-81ef-4ee0-9050-fec3160120fe","Type":"ContainerStarted","Data":"39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3"}
Jul 30 17:15:02.065550 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.065450    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr" event={"ID":"eed33416-81ef-4ee0-9050-fec3160120fe","Type":"ContainerStarted","Data":"9b4549a66ee86727849976802c5e3d322355603709161a3c141119e2b06e29f1"}
Jul 30 17:15:02.065550 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.065480    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:02.066788 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.066767    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v" event={"ID":"39224a0a-5fd7-4e7e-b80a-e3a501be0550","Type":"ContainerStarted","Data":"f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338"}
Jul 30 17:15:02.066901 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.066799    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v" event={"ID":"39224a0a-5fd7-4e7e-b80a-e3a501be0550","Type":"ContainerStarted","Data":"4536d216762d2da3498ab55216429c778866db2b2752222554b00a2d0332639b"}
Jul 30 17:15:02.066901 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.066881    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:02.091764 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.091722    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2" podStartSLOduration=1.091709511 podStartE2EDuration="1.091709511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:15:02.090532217 +0000 UTC m=+2329.692959843" watchObservedRunningTime="2026-07-30 17:15:02.091709511 +0000 UTC m=+2329.694137137"
Jul 30 17:15:02.121755 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.121714    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v" podStartSLOduration=1.534358965 podStartE2EDuration="2.121703621s" totalImagesPullingTime="587.344656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:15:02.119224288 +0000 UTC m=+2329.721651915" watchObservedRunningTime="2026-07-30 17:15:02.121703621 +0000 UTC m=+2329.724131245"
Jul 30 17:15:02.149325 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:02.149286    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr" podStartSLOduration=1.149275457 podStartE2EDuration="1.149275457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:15:02.146645709 +0000 UTC m=+2329.749073336" watchObservedRunningTime="2026-07-30 17:15:02.149275457 +0000 UTC m=+2329.751703081"
Jul 30 17:15:08.076152 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:08.076125    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:13.073670 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:13.073644    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:13.074060 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:13.073915    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:42.297475 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.297441    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"]
Jul 30 17:15:42.300236 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.297711    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v" podUID="39224a0a-5fd7-4e7e-b80a-e3a501be0550" containerName="maas-api" containerID="cri-o://f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338" gracePeriod=30
Jul 30 17:15:42.353207 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.353173    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"]
Jul 30 17:15:42.353453 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.353426    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr" podUID="eed33416-81ef-4ee0-9050-fec3160120fe" containerName="payload-processing" containerID="cri-o://39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3" gracePeriod=30
Jul 30 17:15:42.359575 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.359552    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"]
Jul 30 17:15:42.359797 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.359776    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2" podUID="ac7ca78b-3eff-45dc-972f-65433935ad77" containerName="payload-pre-processing" containerID="cri-o://114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084" gracePeriod=30
Jul 30 17:15:42.622179 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.622157    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:42.636073 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.636053    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:42.643260 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.643231    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:42.736648 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.736623    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmhdn\" (UniqueName: \"kubernetes.io/projected/eed33416-81ef-4ee0-9050-fec3160120fe-kube-api-access-rmhdn\") pod \"eed33416-81ef-4ee0-9050-fec3160120fe\" (UID: \"eed33416-81ef-4ee0-9050-fec3160120fe\") "
Jul 30 17:15:42.736782 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.736659    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94wfv\" (UniqueName: \"kubernetes.io/projected/ac7ca78b-3eff-45dc-972f-65433935ad77-kube-api-access-94wfv\") pod \"ac7ca78b-3eff-45dc-972f-65433935ad77\" (UID: \"ac7ca78b-3eff-45dc-972f-65433935ad77\") "
Jul 30 17:15:42.736782 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.736679    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed33416-81ef-4ee0-9050-fec3160120fe-plugins-config-volume\") pod \"eed33416-81ef-4ee0-9050-fec3160120fe\" (UID: \"eed33416-81ef-4ee0-9050-fec3160120fe\") "
Jul 30 17:15:42.736782 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.736727    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdf9c\" (UniqueName: \"kubernetes.io/projected/39224a0a-5fd7-4e7e-b80a-e3a501be0550-kube-api-access-sdf9c\") pod \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") "
Jul 30 17:15:42.736915 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.736787    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39224a0a-5fd7-4e7e-b80a-e3a501be0550-openshift-service-ca\") pod \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") "
Jul 30 17:15:42.736915 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.736803    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39224a0a-5fd7-4e7e-b80a-e3a501be0550-maas-api-tls\") pod \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\" (UID: \"39224a0a-5fd7-4e7e-b80a-e3a501be0550\") "
Jul 30 17:15:42.736915 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.736836    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7ca78b-3eff-45dc-972f-65433935ad77-plugins-config-volume\") pod \"ac7ca78b-3eff-45dc-972f-65433935ad77\" (UID: \"ac7ca78b-3eff-45dc-972f-65433935ad77\") "
Jul 30 17:15:42.737468 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.737164    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39224a0a-5fd7-4e7e-b80a-e3a501be0550-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39224a0a-5fd7-4e7e-b80a-e3a501be0550" (UID: "39224a0a-5fd7-4e7e-b80a-e3a501be0550"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:15:42.737468 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.737373    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac7ca78b-3eff-45dc-972f-65433935ad77-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac7ca78b-3eff-45dc-972f-65433935ad77" (UID: "ac7ca78b-3eff-45dc-972f-65433935ad77"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:15:42.737468 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.737422    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eed33416-81ef-4ee0-9050-fec3160120fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eed33416-81ef-4ee0-9050-fec3160120fe" (UID: "eed33416-81ef-4ee0-9050-fec3160120fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:15:42.738741 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.738714    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eed33416-81ef-4ee0-9050-fec3160120fe-kube-api-access-rmhdn" (OuterVolumeSpecName: "kube-api-access-rmhdn") pod "eed33416-81ef-4ee0-9050-fec3160120fe" (UID: "eed33416-81ef-4ee0-9050-fec3160120fe"). InnerVolumeSpecName "kube-api-access-rmhdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:15:42.738741 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.738728    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac7ca78b-3eff-45dc-972f-65433935ad77-kube-api-access-94wfv" (OuterVolumeSpecName: "kube-api-access-94wfv") pod "ac7ca78b-3eff-45dc-972f-65433935ad77" (UID: "ac7ca78b-3eff-45dc-972f-65433935ad77"). InnerVolumeSpecName "kube-api-access-94wfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:15:42.738741 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.738713    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39224a0a-5fd7-4e7e-b80a-e3a501be0550-kube-api-access-sdf9c" (OuterVolumeSpecName: "kube-api-access-sdf9c") pod "39224a0a-5fd7-4e7e-b80a-e3a501be0550" (UID: "39224a0a-5fd7-4e7e-b80a-e3a501be0550"). InnerVolumeSpecName "kube-api-access-sdf9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:15:42.738901 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.738791    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39224a0a-5fd7-4e7e-b80a-e3a501be0550-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39224a0a-5fd7-4e7e-b80a-e3a501be0550" (UID: "39224a0a-5fd7-4e7e-b80a-e3a501be0550"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:15:42.838120 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.838069    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdf9c\" (UniqueName: \"kubernetes.io/projected/39224a0a-5fd7-4e7e-b80a-e3a501be0550-kube-api-access-sdf9c\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:15:42.838120 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.838089    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39224a0a-5fd7-4e7e-b80a-e3a501be0550-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:15:42.838120 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.838099    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39224a0a-5fd7-4e7e-b80a-e3a501be0550-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:15:42.838120 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.838108    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7ca78b-3eff-45dc-972f-65433935ad77-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:15:42.838120 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.838115    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmhdn\" (UniqueName: \"kubernetes.io/projected/eed33416-81ef-4ee0-9050-fec3160120fe-kube-api-access-rmhdn\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:15:42.838120 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.838123    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94wfv\" (UniqueName: \"kubernetes.io/projected/ac7ca78b-3eff-45dc-972f-65433935ad77-kube-api-access-94wfv\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:15:42.838344 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:42.838131    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed33416-81ef-4ee0-9050-fec3160120fe-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:15:43.257038 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.257006    2569 generic.go:356] "Generic (PLEG): container finished" podID="39224a0a-5fd7-4e7e-b80a-e3a501be0550" containerID="f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338" exitCode=0
Jul 30 17:15:43.257158 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.257043    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v" event={"ID":"39224a0a-5fd7-4e7e-b80a-e3a501be0550","Type":"ContainerDied","Data":"f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338"}
Jul 30 17:15:43.257158 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.257094    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v" event={"ID":"39224a0a-5fd7-4e7e-b80a-e3a501be0550","Type":"ContainerDied","Data":"4536d216762d2da3498ab55216429c778866db2b2752222554b00a2d0332639b"}
Jul 30 17:15:43.257158 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.257116    2569 scope.go:117] "RemoveContainer" containerID="f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338"
Jul 30 17:15:43.257158 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.257051    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"
Jul 30 17:15:43.258518 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.258495    2569 generic.go:356] "Generic (PLEG): container finished" podID="ac7ca78b-3eff-45dc-972f-65433935ad77" containerID="114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084" exitCode=0
Jul 30 17:15:43.258636 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.258577    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2" event={"ID":"ac7ca78b-3eff-45dc-972f-65433935ad77","Type":"ContainerDied","Data":"114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084"}
Jul 30 17:15:43.258636 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.258593    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"
Jul 30 17:15:43.258760 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.258644    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2" event={"ID":"ac7ca78b-3eff-45dc-972f-65433935ad77","Type":"ContainerDied","Data":"4b3df2f3fe6987e21bcabcb90408fce05fc74c3f9e173b1e933b9ed4ec2683e0"}
Jul 30 17:15:43.260122 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.260085    2569 generic.go:356] "Generic (PLEG): container finished" podID="eed33416-81ef-4ee0-9050-fec3160120fe" containerID="39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3" exitCode=0
Jul 30 17:15:43.260239 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.260141    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"
Jul 30 17:15:43.260239 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.260168    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr" event={"ID":"eed33416-81ef-4ee0-9050-fec3160120fe","Type":"ContainerDied","Data":"39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3"}
Jul 30 17:15:43.260239 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.260212    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr" event={"ID":"eed33416-81ef-4ee0-9050-fec3160120fe","Type":"ContainerDied","Data":"9b4549a66ee86727849976802c5e3d322355603709161a3c141119e2b06e29f1"}
Jul 30 17:15:43.266712 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.266693    2569 scope.go:117] "RemoveContainer" containerID="f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338"
Jul 30 17:15:43.266982 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:15:43.266963    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338\": container with ID starting with f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338 not found: ID does not exist" containerID="f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338"
Jul 30 17:15:43.267072 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.266987    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338"} err="failed to get container status \"f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338\": rpc error: code = NotFound desc = could not find container \"f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338\": container with ID starting with f9777efa700169af238f6a1fa0301cab452f488f024a11a1c9d9428ba1fa5338 not found: ID does not exist"
Jul 30 17:15:43.267072 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.267016    2569 scope.go:117] "RemoveContainer" containerID="114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084"
Jul 30 17:15:43.274366 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.274349    2569 scope.go:117] "RemoveContainer" containerID="114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084"
Jul 30 17:15:43.274627 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:15:43.274595    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084\": container with ID starting with 114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084 not found: ID does not exist" containerID="114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084"
Jul 30 17:15:43.274699 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.274634    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084"} err="failed to get container status \"114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084\": rpc error: code = NotFound desc = could not find container \"114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084\": container with ID starting with 114b82ba74c9a7fa7c1fc75dda5179843caf68a6a72d198ef6986e9c5489b084 not found: ID does not exist"
Jul 30 17:15:43.274699 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.274649    2569 scope.go:117] "RemoveContainer" containerID="39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3"
Jul 30 17:15:43.281345 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.281331    2569 scope.go:117] "RemoveContainer" containerID="39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3"
Jul 30 17:15:43.281557 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:15:43.281539    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3\": container with ID starting with 39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3 not found: ID does not exist" containerID="39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3"
Jul 30 17:15:43.281654 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.281563    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3"} err="failed to get container status \"39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3\": rpc error: code = NotFound desc = could not find container \"39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3\": container with ID starting with 39fe5980291b56dd8e29239b1c9f0b8ba345955932fd7c6bbfb1e6ae34c8eef3 not found: ID does not exist"
Jul 30 17:15:43.287353 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.287333    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"]
Jul 30 17:15:43.291963 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.291943    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f2963a-65bb7cd5bf-ph46v"]
Jul 30 17:15:43.302430 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.302412    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"]
Jul 30 17:15:43.306964 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.306946    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e6f2963a-6cc69b5646-gtqtr"]
Jul 30 17:15:43.317138 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.317120    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"]
Jul 30 17:15:43.319705 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:43.319682    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e6f2963a-5dd7b96696-d2xk2"]
Jul 30 17:15:45.006204 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:45.006166    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39224a0a-5fd7-4e7e-b80a-e3a501be0550" path="/var/lib/kubelet/pods/39224a0a-5fd7-4e7e-b80a-e3a501be0550/volumes"
Jul 30 17:15:45.006582 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:45.006562    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac7ca78b-3eff-45dc-972f-65433935ad77" path="/var/lib/kubelet/pods/ac7ca78b-3eff-45dc-972f-65433935ad77/volumes"
Jul 30 17:15:45.007003 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:15:45.006985    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eed33416-81ef-4ee0-9050-fec3160120fe" path="/var/lib/kubelet/pods/eed33416-81ef-4ee0-9050-fec3160120fe/volumes"
Jul 30 17:18:08.438786 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.438747    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"]
Jul 30 17:18:08.439338 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.439321    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eed33416-81ef-4ee0-9050-fec3160120fe" containerName="payload-processing"
Jul 30 17:18:08.439408 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.439342    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eed33416-81ef-4ee0-9050-fec3160120fe" containerName="payload-processing"
Jul 30 17:18:08.439408 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.439363    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39224a0a-5fd7-4e7e-b80a-e3a501be0550" containerName="maas-api"
Jul 30 17:18:08.439408 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.439371    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39224a0a-5fd7-4e7e-b80a-e3a501be0550" containerName="maas-api"
Jul 30 17:18:08.439408 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.439381    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac7ca78b-3eff-45dc-972f-65433935ad77" containerName="payload-pre-processing"
Jul 30 17:18:08.439408 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.439390    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac7ca78b-3eff-45dc-972f-65433935ad77" containerName="payload-pre-processing"
Jul 30 17:18:08.439588 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.439483    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eed33416-81ef-4ee0-9050-fec3160120fe" containerName="payload-processing"
Jul 30 17:18:08.439588 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.439498    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39224a0a-5fd7-4e7e-b80a-e3a501be0550" containerName="maas-api"
Jul 30 17:18:08.439588 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.439508    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac7ca78b-3eff-45dc-972f-65433935ad77" containerName="payload-pre-processing"
Jul 30 17:18:08.442949 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.442927    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:08.445784 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.445748    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8eefd8d1-serving-cert\""
Jul 30 17:18:08.454270 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.454248    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"]
Jul 30 17:18:08.482671 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.482647    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c85147f-d782-438b-82e9-019d0622fe69-maas-api-tls\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:08.482791 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.482682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn9xf\" (UniqueName: \"kubernetes.io/projected/5c85147f-d782-438b-82e9-019d0622fe69-kube-api-access-hn9xf\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:08.482791 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.482728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c85147f-d782-438b-82e9-019d0622fe69-openshift-service-ca\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:08.583889 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.583865    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c85147f-d782-438b-82e9-019d0622fe69-maas-api-tls\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:08.584003 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.583901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn9xf\" (UniqueName: \"kubernetes.io/projected/5c85147f-d782-438b-82e9-019d0622fe69-kube-api-access-hn9xf\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:08.584003 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.583939    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c85147f-d782-438b-82e9-019d0622fe69-openshift-service-ca\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:08.584079 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:18:08.584002    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-serving-cert: secret "maas-api-e2e-mt-8eefd8d1-serving-cert" not found
Jul 30 17:18:08.584079 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:18:08.584070    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c85147f-d782-438b-82e9-019d0622fe69-maas-api-tls podName:5c85147f-d782-438b-82e9-019d0622fe69 nodeName:}" failed. No retries permitted until 2026-07-30 17:18:09.084053499 +0000 UTC m=+2516.686481104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5c85147f-d782-438b-82e9-019d0622fe69-maas-api-tls") pod "maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5" (UID: "5c85147f-d782-438b-82e9-019d0622fe69") : secret "maas-api-e2e-mt-8eefd8d1-serving-cert" not found
Jul 30 17:18:08.584508 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.584493    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c85147f-d782-438b-82e9-019d0622fe69-openshift-service-ca\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:08.594681 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:08.594658    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn9xf\" (UniqueName: \"kubernetes.io/projected/5c85147f-d782-438b-82e9-019d0622fe69-kube-api-access-hn9xf\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:09.085482 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.085445    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"]
Jul 30 17:18:09.087839 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.087810    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c85147f-d782-438b-82e9-019d0622fe69-maas-api-tls\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:09.089095 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.089081    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:18:09.090172 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.090151    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c85147f-d782-438b-82e9-019d0622fe69-maas-api-tls\") pod \"maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:09.091640 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.091618    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8eefd8d1-dockercfg-bfpb4\""
Jul 30 17:18:09.091727 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.091683    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8eefd8d1\""
Jul 30 17:18:09.096393 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.096373    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"]
Jul 30 17:18:09.099906 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.099887    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"]
Jul 30 17:18:09.100149 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.099979    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:09.116330 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.116309    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"]
Jul 30 17:18:09.188557 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.188534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36343aef-3f48-4721-a983-e39565c1b5e5-plugins-config-volume\") pod \"payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz\" (UID: \"36343aef-3f48-4721-a983-e39565c1b5e5\") " pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:09.188686 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.188565    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhzmv\" (UniqueName: \"kubernetes.io/projected/5c6ae398-f462-4477-81e8-6f3adc01367d-kube-api-access-mhzmv\") pod \"payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n\" (UID: \"5c6ae398-f462-4477-81e8-6f3adc01367d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:18:09.188686 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.188662    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f25j\" (UniqueName: \"kubernetes.io/projected/36343aef-3f48-4721-a983-e39565c1b5e5-kube-api-access-4f25j\") pod \"payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz\" (UID: \"36343aef-3f48-4721-a983-e39565c1b5e5\") " pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:09.188768 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.188714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c6ae398-f462-4477-81e8-6f3adc01367d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n\" (UID: \"5c6ae398-f462-4477-81e8-6f3adc01367d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:18:09.289359 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.289329    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f25j\" (UniqueName: \"kubernetes.io/projected/36343aef-3f48-4721-a983-e39565c1b5e5-kube-api-access-4f25j\") pod \"payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz\" (UID: \"36343aef-3f48-4721-a983-e39565c1b5e5\") " pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:09.289515 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.289376    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c6ae398-f462-4477-81e8-6f3adc01367d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n\" (UID: \"5c6ae398-f462-4477-81e8-6f3adc01367d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:18:09.289515 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.289402    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36343aef-3f48-4721-a983-e39565c1b5e5-plugins-config-volume\") pod \"payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz\" (UID: \"36343aef-3f48-4721-a983-e39565c1b5e5\") " pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:09.289515 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.289419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhzmv\" (UniqueName: \"kubernetes.io/projected/5c6ae398-f462-4477-81e8-6f3adc01367d-kube-api-access-mhzmv\") pod \"payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n\" (UID: \"5c6ae398-f462-4477-81e8-6f3adc01367d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:18:09.290083 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.290063    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c6ae398-f462-4477-81e8-6f3adc01367d-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n\" (UID: \"5c6ae398-f462-4477-81e8-6f3adc01367d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:18:09.290133 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.290071    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36343aef-3f48-4721-a983-e39565c1b5e5-plugins-config-volume\") pod \"payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz\" (UID: \"36343aef-3f48-4721-a983-e39565c1b5e5\") " pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:09.301077 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.301045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f25j\" (UniqueName: \"kubernetes.io/projected/36343aef-3f48-4721-a983-e39565c1b5e5-kube-api-access-4f25j\") pod \"payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz\" (UID: \"36343aef-3f48-4721-a983-e39565c1b5e5\") " pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:09.301259 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.301241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhzmv\" (UniqueName: \"kubernetes.io/projected/5c6ae398-f462-4477-81e8-6f3adc01367d-kube-api-access-mhzmv\") pod \"payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n\" (UID: \"5c6ae398-f462-4477-81e8-6f3adc01367d\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:18:09.356564 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.356509    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:09.409813 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.409766    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:18:09.415278 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.415251    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:09.501754 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.501730    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"]
Jul 30 17:18:09.503213 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:18:09.503185    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c85147f_d782_438b_82e9_019d0622fe69.slice/crio-5063dd3f77911e77a9a60e2ad8fc977e6afbb9881b64ff21b35ec47bd08cf46c WatchSource:0}: Error finding container 5063dd3f77911e77a9a60e2ad8fc977e6afbb9881b64ff21b35ec47bd08cf46c: Status 404 returned error can't find the container with id 5063dd3f77911e77a9a60e2ad8fc977e6afbb9881b64ff21b35ec47bd08cf46c
Jul 30 17:18:09.504690 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.504673    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:18:09.565493 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.565465    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"]
Jul 30 17:18:09.566567 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:18:09.566539    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c6ae398_f462_4477_81e8_6f3adc01367d.slice/crio-b07aa6e34f38020178da779602a1f268a4da8b7e8586c22a0a9da4d3518ba779 WatchSource:0}: Error finding container b07aa6e34f38020178da779602a1f268a4da8b7e8586c22a0a9da4d3518ba779: Status 404 returned error can't find the container with id b07aa6e34f38020178da779602a1f268a4da8b7e8586c22a0a9da4d3518ba779
Jul 30 17:18:09.583259 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.583235    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"]
Jul 30 17:18:09.584596 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:18:09.584575    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36343aef_3f48_4721_a983_e39565c1b5e5.slice/crio-e8e8a83345f70e24525aea2218ee270997b92922cd9cd018dd61ed8f1f1309fe WatchSource:0}: Error finding container e8e8a83345f70e24525aea2218ee270997b92922cd9cd018dd61ed8f1f1309fe: Status 404 returned error can't find the container with id e8e8a83345f70e24525aea2218ee270997b92922cd9cd018dd61ed8f1f1309fe
Jul 30 17:18:09.934564 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.934530    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz" event={"ID":"36343aef-3f48-4721-a983-e39565c1b5e5","Type":"ContainerStarted","Data":"ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572"}
Jul 30 17:18:09.934757 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.934695    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:09.934757 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.934733    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz" event={"ID":"36343aef-3f48-4721-a983-e39565c1b5e5","Type":"ContainerStarted","Data":"e8e8a83345f70e24525aea2218ee270997b92922cd9cd018dd61ed8f1f1309fe"}
Jul 30 17:18:09.936072 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.936052    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n" event={"ID":"5c6ae398-f462-4477-81e8-6f3adc01367d","Type":"ContainerStarted","Data":"0ae3d922021211f5d569e0a10e4429c86a6492441c28187e3aa8986ca145fbb0"}
Jul 30 17:18:09.936182 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.936089    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n" event={"ID":"5c6ae398-f462-4477-81e8-6f3adc01367d","Type":"ContainerStarted","Data":"b07aa6e34f38020178da779602a1f268a4da8b7e8586c22a0a9da4d3518ba779"}
Jul 30 17:18:09.936182 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.936116    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:18:09.937195 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.937176    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5" event={"ID":"5c85147f-d782-438b-82e9-019d0622fe69","Type":"ContainerStarted","Data":"5063dd3f77911e77a9a60e2ad8fc977e6afbb9881b64ff21b35ec47bd08cf46c"}
Jul 30 17:18:09.968293 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.968247    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz" podStartSLOduration=0.968233893 podStartE2EDuration="968.233893ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:18:09.965869236 +0000 UTC m=+2517.568296860" watchObservedRunningTime="2026-07-30 17:18:09.968233893 +0000 UTC m=+2517.570661518"
Jul 30 17:18:09.998598 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:09.998554    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n" podStartSLOduration=0.998542102 podStartE2EDuration="998.542102ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:18:09.99638469 +0000 UTC m=+2517.598812315" watchObservedRunningTime="2026-07-30 17:18:09.998542102 +0000 UTC m=+2517.600969706"
Jul 30 17:18:10.943081 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:10.943048    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5" event={"ID":"5c85147f-d782-438b-82e9-019d0622fe69","Type":"ContainerStarted","Data":"8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e"}
Jul 30 17:18:10.943514 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:10.943250    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:10.968873 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:10.968821    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5" podStartSLOduration=2.262040149 podStartE2EDuration="2.968804765s" totalImagesPullingTime="706.764616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:18:10.967071694 +0000 UTC m=+2518.569499318" watchObservedRunningTime="2026-07-30 17:18:10.968804765 +0000 UTC m=+2518.571232392"
Jul 30 17:18:16.951780 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:16.951750    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:18:20.945678 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:20.945648    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:18:20.946139 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:18:20.945725    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:19:48.034381 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.034344    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"]
Jul 30 17:19:48.035006 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.034588    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5" podUID="5c85147f-d782-438b-82e9-019d0622fe69" containerName="maas-api" containerID="cri-o://8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e" gracePeriod=30
Jul 30 17:19:48.092108 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.092075    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"]
Jul 30 17:19:48.092580 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.092311    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz" podUID="36343aef-3f48-4721-a983-e39565c1b5e5" containerName="payload-processing" containerID="cri-o://ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572" gracePeriod=30
Jul 30 17:19:48.102491 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.102465    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"]
Jul 30 17:19:48.102693 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.102668    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n" podUID="5c6ae398-f462-4477-81e8-6f3adc01367d" containerName="payload-pre-processing" containerID="cri-o://0ae3d922021211f5d569e0a10e4429c86a6492441c28187e3aa8986ca145fbb0" gracePeriod=30
Jul 30 17:19:48.364458 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.364433    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:19:48.374468 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.374448    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:19:48.381205 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.381184    2569 generic.go:356] "Generic (PLEG): container finished" podID="5c85147f-d782-438b-82e9-019d0622fe69" containerID="8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e" exitCode=0
Jul 30 17:19:48.381314 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.381222    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"
Jul 30 17:19:48.381314 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.381295    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5" event={"ID":"5c85147f-d782-438b-82e9-019d0622fe69","Type":"ContainerDied","Data":"8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e"}
Jul 30 17:19:48.381428 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.381330    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5" event={"ID":"5c85147f-d782-438b-82e9-019d0622fe69","Type":"ContainerDied","Data":"5063dd3f77911e77a9a60e2ad8fc977e6afbb9881b64ff21b35ec47bd08cf46c"}
Jul 30 17:19:48.381428 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.381354    2569 scope.go:117] "RemoveContainer" containerID="8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e"
Jul 30 17:19:48.382861 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.382708    2569 generic.go:356] "Generic (PLEG): container finished" podID="36343aef-3f48-4721-a983-e39565c1b5e5" containerID="ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572" exitCode=0
Jul 30 17:19:48.382861 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.382739    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz" event={"ID":"36343aef-3f48-4721-a983-e39565c1b5e5","Type":"ContainerDied","Data":"ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572"}
Jul 30 17:19:48.382861 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.382761    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"
Jul 30 17:19:48.382861 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.382787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz" event={"ID":"36343aef-3f48-4721-a983-e39565c1b5e5","Type":"ContainerDied","Data":"e8e8a83345f70e24525aea2218ee270997b92922cd9cd018dd61ed8f1f1309fe"}
Jul 30 17:19:48.384666 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.384642    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n" event={"ID":"5c6ae398-f462-4477-81e8-6f3adc01367d","Type":"ContainerDied","Data":"0ae3d922021211f5d569e0a10e4429c86a6492441c28187e3aa8986ca145fbb0"}
Jul 30 17:19:48.384767 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.384646    2569 generic.go:356] "Generic (PLEG): container finished" podID="5c6ae398-f462-4477-81e8-6f3adc01367d" containerID="0ae3d922021211f5d569e0a10e4429c86a6492441c28187e3aa8986ca145fbb0" exitCode=0
Jul 30 17:19:48.391432 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.391409    2569 scope.go:117] "RemoveContainer" containerID="8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e"
Jul 30 17:19:48.391759 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:19:48.391727    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e\": container with ID starting with 8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e not found: ID does not exist" containerID="8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e"
Jul 30 17:19:48.391831 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.391760    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e"} err="failed to get container status \"8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e\": rpc error: code = NotFound desc = could not find container \"8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e\": container with ID starting with 8bbac4be86d9be58782e0e64bab4b4f90cce791fa61202ef0850108c055a040e not found: ID does not exist"
Jul 30 17:19:48.391831 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.391783    2569 scope.go:117] "RemoveContainer" containerID="ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572"
Jul 30 17:19:48.400851 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.400829    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:19:48.401497 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.401483    2569 scope.go:117] "RemoveContainer" containerID="ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572"
Jul 30 17:19:48.401755 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:19:48.401736    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572\": container with ID starting with ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572 not found: ID does not exist" containerID="ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572"
Jul 30 17:19:48.401842 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.401760    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572"} err="failed to get container status \"ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572\": rpc error: code = NotFound desc = could not find container \"ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572\": container with ID starting with ae1c675c6325dce73b67f390b09f8bbfd6781894bdb9a0b304eea78424399572 not found: ID does not exist"
Jul 30 17:19:48.504373 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.504348    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c6ae398-f462-4477-81e8-6f3adc01367d-plugins-config-volume\") pod \"5c6ae398-f462-4477-81e8-6f3adc01367d\" (UID: \"5c6ae398-f462-4477-81e8-6f3adc01367d\") "
Jul 30 17:19:48.504512 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.504387    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhzmv\" (UniqueName: \"kubernetes.io/projected/5c6ae398-f462-4477-81e8-6f3adc01367d-kube-api-access-mhzmv\") pod \"5c6ae398-f462-4477-81e8-6f3adc01367d\" (UID: \"5c6ae398-f462-4477-81e8-6f3adc01367d\") "
Jul 30 17:19:48.504512 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.504431    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c85147f-d782-438b-82e9-019d0622fe69-openshift-service-ca\") pod \"5c85147f-d782-438b-82e9-019d0622fe69\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") "
Jul 30 17:19:48.504649 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.504532    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f25j\" (UniqueName: \"kubernetes.io/projected/36343aef-3f48-4721-a983-e39565c1b5e5-kube-api-access-4f25j\") pod \"36343aef-3f48-4721-a983-e39565c1b5e5\" (UID: \"36343aef-3f48-4721-a983-e39565c1b5e5\") "
Jul 30 17:19:48.504703 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.504642    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c85147f-d782-438b-82e9-019d0622fe69-maas-api-tls\") pod \"5c85147f-d782-438b-82e9-019d0622fe69\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") "
Jul 30 17:19:48.504762 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.504730    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn9xf\" (UniqueName: \"kubernetes.io/projected/5c85147f-d782-438b-82e9-019d0622fe69-kube-api-access-hn9xf\") pod \"5c85147f-d782-438b-82e9-019d0622fe69\" (UID: \"5c85147f-d782-438b-82e9-019d0622fe69\") "
Jul 30 17:19:48.504762 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.504755    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36343aef-3f48-4721-a983-e39565c1b5e5-plugins-config-volume\") pod \"36343aef-3f48-4721-a983-e39565c1b5e5\" (UID: \"36343aef-3f48-4721-a983-e39565c1b5e5\") "
Jul 30 17:19:48.504875 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.504751    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c85147f-d782-438b-82e9-019d0622fe69-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c85147f-d782-438b-82e9-019d0622fe69" (UID: "5c85147f-d782-438b-82e9-019d0622fe69"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:19:48.504955 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.504783    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c6ae398-f462-4477-81e8-6f3adc01367d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5c6ae398-f462-4477-81e8-6f3adc01367d" (UID: "5c6ae398-f462-4477-81e8-6f3adc01367d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:19:48.505130 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.505105    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36343aef-3f48-4721-a983-e39565c1b5e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "36343aef-3f48-4721-a983-e39565c1b5e5" (UID: "36343aef-3f48-4721-a983-e39565c1b5e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:19:48.505244 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.505207    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36343aef-3f48-4721-a983-e39565c1b5e5-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:19:48.505244 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.505225    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c6ae398-f462-4477-81e8-6f3adc01367d-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:19:48.505244 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.505234    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c85147f-d782-438b-82e9-019d0622fe69-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:19:48.506813 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.506791    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36343aef-3f48-4721-a983-e39565c1b5e5-kube-api-access-4f25j" (OuterVolumeSpecName: "kube-api-access-4f25j") pod "36343aef-3f48-4721-a983-e39565c1b5e5" (UID: "36343aef-3f48-4721-a983-e39565c1b5e5"). InnerVolumeSpecName "kube-api-access-4f25j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:19:48.506922 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.506899    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c85147f-d782-438b-82e9-019d0622fe69-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c85147f-d782-438b-82e9-019d0622fe69" (UID: "5c85147f-d782-438b-82e9-019d0622fe69"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:19:48.507026 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.507007    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c85147f-d782-438b-82e9-019d0622fe69-kube-api-access-hn9xf" (OuterVolumeSpecName: "kube-api-access-hn9xf") pod "5c85147f-d782-438b-82e9-019d0622fe69" (UID: "5c85147f-d782-438b-82e9-019d0622fe69"). InnerVolumeSpecName "kube-api-access-hn9xf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:19:48.507067 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.507034    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c6ae398-f462-4477-81e8-6f3adc01367d-kube-api-access-mhzmv" (OuterVolumeSpecName: "kube-api-access-mhzmv") pod "5c6ae398-f462-4477-81e8-6f3adc01367d" (UID: "5c6ae398-f462-4477-81e8-6f3adc01367d"). InnerVolumeSpecName "kube-api-access-mhzmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:19:48.606222 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.606166    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f25j\" (UniqueName: \"kubernetes.io/projected/36343aef-3f48-4721-a983-e39565c1b5e5-kube-api-access-4f25j\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:19:48.606222 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.606188    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c85147f-d782-438b-82e9-019d0622fe69-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:19:48.606222 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.606197    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn9xf\" (UniqueName: \"kubernetes.io/projected/5c85147f-d782-438b-82e9-019d0622fe69-kube-api-access-hn9xf\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:19:48.606222 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.606205    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhzmv\" (UniqueName: \"kubernetes.io/projected/5c6ae398-f462-4477-81e8-6f3adc01367d-kube-api-access-mhzmv\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:19:48.705790 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.705767    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"]
Jul 30 17:19:48.707500 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.707454    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8eefd8d1-7598bb578-9jsvz"]
Jul 30 17:19:48.716997 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.716978    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"]
Jul 30 17:19:48.719393 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:48.719374    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8eefd8d1-688f694566-rxlk5"]
Jul 30 17:19:49.001116 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:49.001088    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36343aef-3f48-4721-a983-e39565c1b5e5" path="/var/lib/kubelet/pods/36343aef-3f48-4721-a983-e39565c1b5e5/volumes"
Jul 30 17:19:49.001471 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:49.001456    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c85147f-d782-438b-82e9-019d0622fe69" path="/var/lib/kubelet/pods/5c85147f-d782-438b-82e9-019d0622fe69/volumes"
Jul 30 17:19:49.390700 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:49.390671    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"
Jul 30 17:19:49.391112 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:49.390668    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n" event={"ID":"5c6ae398-f462-4477-81e8-6f3adc01367d","Type":"ContainerDied","Data":"b07aa6e34f38020178da779602a1f268a4da8b7e8586c22a0a9da4d3518ba779"}
Jul 30 17:19:49.391112 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:49.390816    2569 scope.go:117] "RemoveContainer" containerID="0ae3d922021211f5d569e0a10e4429c86a6492441c28187e3aa8986ca145fbb0"
Jul 30 17:19:49.408118 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:49.408095    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"]
Jul 30 17:19:49.413113 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:49.413093    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8eefd8d1-75844c467b-xpc2n"]
Jul 30 17:19:51.001475 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:19:51.001440    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c6ae398-f462-4477-81e8-6f3adc01367d" path="/var/lib/kubelet/pods/5c6ae398-f462-4477-81e8-6f3adc01367d/volumes"
Jul 30 17:22:00.794421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794386    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"]
Jul 30 17:22:00.794929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794779    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c85147f-d782-438b-82e9-019d0622fe69" containerName="maas-api"
Jul 30 17:22:00.794929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794792    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c85147f-d782-438b-82e9-019d0622fe69" containerName="maas-api"
Jul 30 17:22:00.794929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794802    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36343aef-3f48-4721-a983-e39565c1b5e5" containerName="payload-processing"
Jul 30 17:22:00.794929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794808    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="36343aef-3f48-4721-a983-e39565c1b5e5" containerName="payload-processing"
Jul 30 17:22:00.794929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794820    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c6ae398-f462-4477-81e8-6f3adc01367d" containerName="payload-pre-processing"
Jul 30 17:22:00.794929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794825    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c6ae398-f462-4477-81e8-6f3adc01367d" containerName="payload-pre-processing"
Jul 30 17:22:00.794929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794879    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="36343aef-3f48-4721-a983-e39565c1b5e5" containerName="payload-processing"
Jul 30 17:22:00.794929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794888    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c85147f-d782-438b-82e9-019d0622fe69" containerName="maas-api"
Jul 30 17:22:00.794929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.794895    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c6ae398-f462-4477-81e8-6f3adc01367d" containerName="payload-pre-processing"
Jul 30 17:22:00.797091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.797072    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:00.800793 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.800769    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9f05e7-serving-cert\""
Jul 30 17:22:00.810245 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.810217    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"]
Jul 30 17:22:00.896386 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.896345    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d12a5bb-2f25-4f35-86f4-02cf36216049-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:00.896572 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.896402    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d12a5bb-2f25-4f35-86f4-02cf36216049-maas-api-tls\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:00.896572 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.896452    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-595bn\" (UniqueName: \"kubernetes.io/projected/0d12a5bb-2f25-4f35-86f4-02cf36216049-kube-api-access-595bn\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:00.996974 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.996933    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-595bn\" (UniqueName: \"kubernetes.io/projected/0d12a5bb-2f25-4f35-86f4-02cf36216049-kube-api-access-595bn\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:00.997157 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.997027    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d12a5bb-2f25-4f35-86f4-02cf36216049-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:00.997157 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.997084    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d12a5bb-2f25-4f35-86f4-02cf36216049-maas-api-tls\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:00.997239 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:22:00.997205    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-serving-cert: secret "maas-api-e2e-inf-a-9f05e7-serving-cert" not found
Jul 30 17:22:00.997308 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:22:00.997297    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d12a5bb-2f25-4f35-86f4-02cf36216049-maas-api-tls podName:0d12a5bb-2f25-4f35-86f4-02cf36216049 nodeName:}" failed. No retries permitted until 2026-07-30 17:22:01.497271455 +0000 UTC m=+2749.099699062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d12a5bb-2f25-4f35-86f4-02cf36216049-maas-api-tls") pod "maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr" (UID: "0d12a5bb-2f25-4f35-86f4-02cf36216049") : secret "maas-api-e2e-inf-a-9f05e7-serving-cert" not found
Jul 30 17:22:00.997656 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:00.997635    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d12a5bb-2f25-4f35-86f4-02cf36216049-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:01.006726 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.006697    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-595bn\" (UniqueName: \"kubernetes.io/projected/0d12a5bb-2f25-4f35-86f4-02cf36216049-kube-api-access-595bn\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:01.502784 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.502738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d12a5bb-2f25-4f35-86f4-02cf36216049-maas-api-tls\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:01.505196 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.505166    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d12a5bb-2f25-4f35-86f4-02cf36216049-maas-api-tls\") pod \"maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:01.527085 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.527057    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"]
Jul 30 17:22:01.529338 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.529322    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:01.533943 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.533923    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-9f05e7-dockercfg-9pzk5\""
Jul 30 17:22:01.534490 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.534470    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-9f05e7\""
Jul 30 17:22:01.540688 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.540667    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"]
Jul 30 17:22:01.567295 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.567265    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"]
Jul 30 17:22:01.570019 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.570001    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:01.583473 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.583442    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"]
Jul 30 17:22:01.603760 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.603724    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd3b71d1-515f-47fb-a103-bee705a1c95c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl\" (UID: \"bd3b71d1-515f-47fb-a103-bee705a1c95c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:01.603760 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.603759    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a64f3512-5a22-4d9d-b369-546af190c59d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp\" (UID: \"a64f3512-5a22-4d9d-b369-546af190c59d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:01.603978 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.603818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx784\" (UniqueName: \"kubernetes.io/projected/bd3b71d1-515f-47fb-a103-bee705a1c95c-kube-api-access-qx784\") pod \"payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl\" (UID: \"bd3b71d1-515f-47fb-a103-bee705a1c95c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:01.603978 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.603908    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cpdj\" (UniqueName: \"kubernetes.io/projected/a64f3512-5a22-4d9d-b369-546af190c59d-kube-api-access-2cpdj\") pod \"payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp\" (UID: \"a64f3512-5a22-4d9d-b369-546af190c59d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:01.704952 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.704919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd3b71d1-515f-47fb-a103-bee705a1c95c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl\" (UID: \"bd3b71d1-515f-47fb-a103-bee705a1c95c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:01.704952 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.704953    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a64f3512-5a22-4d9d-b369-546af190c59d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp\" (UID: \"a64f3512-5a22-4d9d-b369-546af190c59d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:01.705171 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.704990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx784\" (UniqueName: \"kubernetes.io/projected/bd3b71d1-515f-47fb-a103-bee705a1c95c-kube-api-access-qx784\") pod \"payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl\" (UID: \"bd3b71d1-515f-47fb-a103-bee705a1c95c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:01.705171 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.705054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cpdj\" (UniqueName: \"kubernetes.io/projected/a64f3512-5a22-4d9d-b369-546af190c59d-kube-api-access-2cpdj\") pod \"payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp\" (UID: \"a64f3512-5a22-4d9d-b369-546af190c59d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:01.705747 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.705728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd3b71d1-515f-47fb-a103-bee705a1c95c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl\" (UID: \"bd3b71d1-515f-47fb-a103-bee705a1c95c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:01.705885 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.705865    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a64f3512-5a22-4d9d-b369-546af190c59d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp\" (UID: \"a64f3512-5a22-4d9d-b369-546af190c59d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:01.707869 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.707850    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:01.714577 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.714550    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx784\" (UniqueName: \"kubernetes.io/projected/bd3b71d1-515f-47fb-a103-bee705a1c95c-kube-api-access-qx784\") pod \"payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl\" (UID: \"bd3b71d1-515f-47fb-a103-bee705a1c95c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:01.714725 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.714706    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cpdj\" (UniqueName: \"kubernetes.io/projected/a64f3512-5a22-4d9d-b369-546af190c59d-kube-api-access-2cpdj\") pod \"payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp\" (UID: \"a64f3512-5a22-4d9d-b369-546af190c59d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:01.840112 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.840076    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:01.845284 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.845262    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"]
Jul 30 17:22:01.849163 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:22:01.849138    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d12a5bb_2f25_4f35_86f4_02cf36216049.slice/crio-6f01ca328e727fe339758425003c8b528abaab29265f50b05ba69c72411bf90a WatchSource:0}: Error finding container 6f01ca328e727fe339758425003c8b528abaab29265f50b05ba69c72411bf90a: Status 404 returned error can't find the container with id 6f01ca328e727fe339758425003c8b528abaab29265f50b05ba69c72411bf90a
Jul 30 17:22:01.879345 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.879308    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:01.980246 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.980220    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"]
Jul 30 17:22:01.982775 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:22:01.982735    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd3b71d1_515f_47fb_a103_bee705a1c95c.slice/crio-df88c4b48e9ae9373407995100a10c54fd5d2a793897906cf3d52937d5c279b1 WatchSource:0}: Error finding container df88c4b48e9ae9373407995100a10c54fd5d2a793897906cf3d52937d5c279b1: Status 404 returned error can't find the container with id df88c4b48e9ae9373407995100a10c54fd5d2a793897906cf3d52937d5c279b1
Jul 30 17:22:01.985721 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:01.985698    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr" event={"ID":"0d12a5bb-2f25-4f35-86f4-02cf36216049","Type":"ContainerStarted","Data":"6f01ca328e727fe339758425003c8b528abaab29265f50b05ba69c72411bf90a"}
Jul 30 17:22:02.229621 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:02.229583    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"]
Jul 30 17:22:02.232199 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:22:02.232171    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda64f3512_5a22_4d9d_b369_546af190c59d.slice/crio-a8034e307ad0addda52f01aa8004c10798137cbfa4028d388a82e9cce8aa8aa7 WatchSource:0}: Error finding container a8034e307ad0addda52f01aa8004c10798137cbfa4028d388a82e9cce8aa8aa7: Status 404 returned error can't find the container with id a8034e307ad0addda52f01aa8004c10798137cbfa4028d388a82e9cce8aa8aa7
Jul 30 17:22:02.991839 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:02.991803    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr" event={"ID":"0d12a5bb-2f25-4f35-86f4-02cf36216049","Type":"ContainerStarted","Data":"cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f"}
Jul 30 17:22:02.992293 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:02.991875    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:02.993371 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:02.993347    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp" event={"ID":"a64f3512-5a22-4d9d-b369-546af190c59d","Type":"ContainerStarted","Data":"049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27"}
Jul 30 17:22:02.993487 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:02.993391    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp" event={"ID":"a64f3512-5a22-4d9d-b369-546af190c59d","Type":"ContainerStarted","Data":"a8034e307ad0addda52f01aa8004c10798137cbfa4028d388a82e9cce8aa8aa7"}
Jul 30 17:22:02.993487 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:02.993418    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:03.000065 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:03.000039    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl" event={"ID":"bd3b71d1-515f-47fb-a103-bee705a1c95c","Type":"ContainerStarted","Data":"65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d"}
Jul 30 17:22:03.000203 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:03.000081    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl" event={"ID":"bd3b71d1-515f-47fb-a103-bee705a1c95c","Type":"ContainerStarted","Data":"df88c4b48e9ae9373407995100a10c54fd5d2a793897906cf3d52937d5c279b1"}
Jul 30 17:22:03.000203 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:03.000109    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:03.009390 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:03.009348    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr" podStartSLOduration=2.575502382 podStartE2EDuration="3.009333726s" totalImagesPullingTime="433.831344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:22:03.008065008 +0000 UTC m=+2750.610492657" watchObservedRunningTime="2026-07-30 17:22:03.009333726 +0000 UTC m=+2750.611761351"
Jul 30 17:22:03.023966 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:03.023923    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl" podStartSLOduration=2.023909155 podStartE2EDuration="2.023909155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:22:03.022790393 +0000 UTC m=+2750.625218020" watchObservedRunningTime="2026-07-30 17:22:03.023909155 +0000 UTC m=+2750.626336780"
Jul 30 17:22:09.011033 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:09.011005    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:22:09.028908 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:09.028858    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp" podStartSLOduration=8.028843311 podStartE2EDuration="8.028843311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:22:03.038434778 +0000 UTC m=+2750.640862404" watchObservedRunningTime="2026-07-30 17:22:09.028843311 +0000 UTC m=+2756.631270989"
Jul 30 17:22:14.008003 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:14.007968    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:22:14.008400 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:14.008150    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:22:29.909805 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:29.909765    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"]
Jul 30 17:22:29.913228 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:29.913207    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:29.915871 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:29.915845    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-4c7e9f-serving-cert\""
Jul 30 17:22:29.923343 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:29.923316    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"]
Jul 30 17:22:30.051427 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.051393    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1e646fc-d2ea-406d-8286-7f66a12c4015-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.051680 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.051555    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1e646fc-d2ea-406d-8286-7f66a12c4015-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.051680 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.051598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pkf9\" (UniqueName: \"kubernetes.io/projected/b1e646fc-d2ea-406d-8286-7f66a12c4015-kube-api-access-4pkf9\") pod \"maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.152562 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.152528    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1e646fc-d2ea-406d-8286-7f66a12c4015-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.152562 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.152564    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkf9\" (UniqueName: \"kubernetes.io/projected/b1e646fc-d2ea-406d-8286-7f66a12c4015-kube-api-access-4pkf9\") pod \"maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.152844 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.152596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1e646fc-d2ea-406d-8286-7f66a12c4015-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.153262 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.153239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1e646fc-d2ea-406d-8286-7f66a12c4015-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.155042 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.155020    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1e646fc-d2ea-406d-8286-7f66a12c4015-maas-api-tls\") pod \"maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.160859 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.160801    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pkf9\" (UniqueName: \"kubernetes.io/projected/b1e646fc-d2ea-406d-8286-7f66a12c4015-kube-api-access-4pkf9\") pod \"maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.225009 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.224968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:30.366476 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.366142    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"]
Jul 30 17:22:30.371353 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:22:30.371321    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e646fc_d2ea_406d_8286_7f66a12c4015.slice/crio-4b93224576b8fc2fdfd1213386a8a520294346c2b44311ea6ed484f55f6511e8 WatchSource:0}: Error finding container 4b93224576b8fc2fdfd1213386a8a520294346c2b44311ea6ed484f55f6511e8: Status 404 returned error can't find the container with id 4b93224576b8fc2fdfd1213386a8a520294346c2b44311ea6ed484f55f6511e8
Jul 30 17:22:30.566843 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.566757    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"]
Jul 30 17:22:30.571376 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.571352    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:22:30.574174 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.574151    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-4c7e9f\""
Jul 30 17:22:30.574317 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.574300    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-4c7e9f-dockercfg-v8qc6\""
Jul 30 17:22:30.581196 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.581149    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"]
Jul 30 17:22:30.656888 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.656844    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2dvv\" (UniqueName: \"kubernetes.io/projected/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-kube-api-access-j2dvv\") pod \"payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m\" (UID: \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:22:30.657072 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.656989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m\" (UID: \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:22:30.757547 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.757519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m\" (UID: \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:22:30.757700 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.757573    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2dvv\" (UniqueName: \"kubernetes.io/projected/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-kube-api-access-j2dvv\") pod \"payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m\" (UID: \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:22:30.758245 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.758226    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m\" (UID: \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:22:30.765812 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.765788    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2dvv\" (UniqueName: \"kubernetes.io/projected/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-kube-api-access-j2dvv\") pod \"payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m\" (UID: \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:22:30.883525 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:30.883433    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:22:31.017099 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:31.017074    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"]
Jul 30 17:22:31.019557 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:22:31.019531    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e6e55e_9a1b_4e24_bfd9_13ba9665cd56.slice/crio-dae41ff6d963e3022ad6e5cd4d9d776d02449167cd7e262932044be2c10e3f42 WatchSource:0}: Error finding container dae41ff6d963e3022ad6e5cd4d9d776d02449167cd7e262932044be2c10e3f42: Status 404 returned error can't find the container with id dae41ff6d963e3022ad6e5cd4d9d776d02449167cd7e262932044be2c10e3f42
Jul 30 17:22:31.131507 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:31.131472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m" event={"ID":"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56","Type":"ContainerStarted","Data":"72c9bf3ee23ebb775bca7c0c6869a7bf98651ded482172cbcb7b33e684db0e12"}
Jul 30 17:22:31.131711 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:31.131535    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m" event={"ID":"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56","Type":"ContainerStarted","Data":"dae41ff6d963e3022ad6e5cd4d9d776d02449167cd7e262932044be2c10e3f42"}
Jul 30 17:22:31.131711 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:31.131578    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:22:31.132983 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:31.132961    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5" event={"ID":"b1e646fc-d2ea-406d-8286-7f66a12c4015","Type":"ContainerStarted","Data":"3fe10b320bd26d86b3d987348a4792536fa06743650888f8369b9e130575db40"}
Jul 30 17:22:31.133081 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:31.132996    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5" event={"ID":"b1e646fc-d2ea-406d-8286-7f66a12c4015","Type":"ContainerStarted","Data":"4b93224576b8fc2fdfd1213386a8a520294346c2b44311ea6ed484f55f6511e8"}
Jul 30 17:22:31.133081 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:31.133044    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:31.159734 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:31.159684    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m" podStartSLOduration=1.159668515 podStartE2EDuration="1.159668515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:22:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:22:31.15726892 +0000 UTC m=+2778.759696546" watchObservedRunningTime="2026-07-30 17:22:31.159668515 +0000 UTC m=+2778.762096140"
Jul 30 17:22:31.177329 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:31.177278    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5" podStartSLOduration=1.8004092269999998 podStartE2EDuration="2.177261382s" totalImagesPullingTime="376.852155ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:22:31.175460578 +0000 UTC m=+2778.777888195" watchObservedRunningTime="2026-07-30 17:22:31.177261382 +0000 UTC m=+2778.779689007"
Jul 30 17:22:37.142915 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:37.142838    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:22:42.140262 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:22:42.140231    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:24:35.777547 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:35.777510    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"]
Jul 30 17:24:35.778014 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:35.777723    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr" podUID="0d12a5bb-2f25-4f35-86f4-02cf36216049" containerName="maas-api" containerID="cri-o://cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f" gracePeriod=30
Jul 30 17:24:35.850049 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:35.850011    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"]
Jul 30 17:24:35.850302 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:35.850269    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp" podUID="a64f3512-5a22-4d9d-b369-546af190c59d" containerName="payload-processing" containerID="cri-o://049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27" gracePeriod=30
Jul 30 17:24:35.860058 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:35.860033    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"]
Jul 30 17:24:35.860274 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:35.860236    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl" podUID="bd3b71d1-515f-47fb-a103-bee705a1c95c" containerName="payload-pre-processing" containerID="cri-o://65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d" gracePeriod=30
Jul 30 17:24:36.142491 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.142465    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:24:36.155130 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.153873    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:24:36.166255 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.166228    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:24:36.251235 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.251201    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d12a5bb-2f25-4f35-86f4-02cf36216049-openshift-service-ca\") pod \"0d12a5bb-2f25-4f35-86f4-02cf36216049\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") "
Jul 30 17:24:36.251235 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.251240    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx784\" (UniqueName: \"kubernetes.io/projected/bd3b71d1-515f-47fb-a103-bee705a1c95c-kube-api-access-qx784\") pod \"bd3b71d1-515f-47fb-a103-bee705a1c95c\" (UID: \"bd3b71d1-515f-47fb-a103-bee705a1c95c\") "
Jul 30 17:24:36.251485 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.251265    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a64f3512-5a22-4d9d-b369-546af190c59d-plugins-config-volume\") pod \"a64f3512-5a22-4d9d-b369-546af190c59d\" (UID: \"a64f3512-5a22-4d9d-b369-546af190c59d\") "
Jul 30 17:24:36.251485 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.251332    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d12a5bb-2f25-4f35-86f4-02cf36216049-maas-api-tls\") pod \"0d12a5bb-2f25-4f35-86f4-02cf36216049\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") "
Jul 30 17:24:36.251485 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.251352    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd3b71d1-515f-47fb-a103-bee705a1c95c-plugins-config-volume\") pod \"bd3b71d1-515f-47fb-a103-bee705a1c95c\" (UID: \"bd3b71d1-515f-47fb-a103-bee705a1c95c\") "
Jul 30 17:24:36.251485 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.251381    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-595bn\" (UniqueName: \"kubernetes.io/projected/0d12a5bb-2f25-4f35-86f4-02cf36216049-kube-api-access-595bn\") pod \"0d12a5bb-2f25-4f35-86f4-02cf36216049\" (UID: \"0d12a5bb-2f25-4f35-86f4-02cf36216049\") "
Jul 30 17:24:36.251485 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.251435    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cpdj\" (UniqueName: \"kubernetes.io/projected/a64f3512-5a22-4d9d-b369-546af190c59d-kube-api-access-2cpdj\") pod \"a64f3512-5a22-4d9d-b369-546af190c59d\" (UID: \"a64f3512-5a22-4d9d-b369-546af190c59d\") "
Jul 30 17:24:36.251796 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.251758    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a64f3512-5a22-4d9d-b369-546af190c59d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a64f3512-5a22-4d9d-b369-546af190c59d" (UID: "a64f3512-5a22-4d9d-b369-546af190c59d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:24:36.251855 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.251797    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd3b71d1-515f-47fb-a103-bee705a1c95c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd3b71d1-515f-47fb-a103-bee705a1c95c" (UID: "bd3b71d1-515f-47fb-a103-bee705a1c95c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:24:36.252169 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.252140    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d12a5bb-2f25-4f35-86f4-02cf36216049-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d12a5bb-2f25-4f35-86f4-02cf36216049" (UID: "0d12a5bb-2f25-4f35-86f4-02cf36216049"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:24:36.253549 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.253528    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d12a5bb-2f25-4f35-86f4-02cf36216049-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d12a5bb-2f25-4f35-86f4-02cf36216049" (UID: "0d12a5bb-2f25-4f35-86f4-02cf36216049"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:24:36.253896 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.253874    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd3b71d1-515f-47fb-a103-bee705a1c95c-kube-api-access-qx784" (OuterVolumeSpecName: "kube-api-access-qx784") pod "bd3b71d1-515f-47fb-a103-bee705a1c95c" (UID: "bd3b71d1-515f-47fb-a103-bee705a1c95c"). InnerVolumeSpecName "kube-api-access-qx784". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:24:36.254041 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.254023    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a64f3512-5a22-4d9d-b369-546af190c59d-kube-api-access-2cpdj" (OuterVolumeSpecName: "kube-api-access-2cpdj") pod "a64f3512-5a22-4d9d-b369-546af190c59d" (UID: "a64f3512-5a22-4d9d-b369-546af190c59d"). InnerVolumeSpecName "kube-api-access-2cpdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:24:36.254085 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.254049    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d12a5bb-2f25-4f35-86f4-02cf36216049-kube-api-access-595bn" (OuterVolumeSpecName: "kube-api-access-595bn") pod "0d12a5bb-2f25-4f35-86f4-02cf36216049" (UID: "0d12a5bb-2f25-4f35-86f4-02cf36216049"). InnerVolumeSpecName "kube-api-access-595bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:24:36.352233 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.352177    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d12a5bb-2f25-4f35-86f4-02cf36216049-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:36.352233 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.352200    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx784\" (UniqueName: \"kubernetes.io/projected/bd3b71d1-515f-47fb-a103-bee705a1c95c-kube-api-access-qx784\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:36.352233 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.352210    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a64f3512-5a22-4d9d-b369-546af190c59d-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:36.352233 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.352218    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d12a5bb-2f25-4f35-86f4-02cf36216049-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:36.352233 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.352228    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd3b71d1-515f-47fb-a103-bee705a1c95c-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:36.352434 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.352237    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-595bn\" (UniqueName: \"kubernetes.io/projected/0d12a5bb-2f25-4f35-86f4-02cf36216049-kube-api-access-595bn\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:36.352434 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.352246    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cpdj\" (UniqueName: \"kubernetes.io/projected/a64f3512-5a22-4d9d-b369-546af190c59d-kube-api-access-2cpdj\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:36.689117 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.689093    2569 generic.go:356] "Generic (PLEG): container finished" podID="0d12a5bb-2f25-4f35-86f4-02cf36216049" containerID="cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f" exitCode=0
Jul 30 17:24:36.689267 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.689142    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"
Jul 30 17:24:36.689267 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.689160    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr" event={"ID":"0d12a5bb-2f25-4f35-86f4-02cf36216049","Type":"ContainerDied","Data":"cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f"}
Jul 30 17:24:36.689267 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.689211    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr" event={"ID":"0d12a5bb-2f25-4f35-86f4-02cf36216049","Type":"ContainerDied","Data":"6f01ca328e727fe339758425003c8b528abaab29265f50b05ba69c72411bf90a"}
Jul 30 17:24:36.689267 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.689235    2569 scope.go:117] "RemoveContainer" containerID="cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f"
Jul 30 17:24:36.690393 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.690373    2569 generic.go:356] "Generic (PLEG): container finished" podID="a64f3512-5a22-4d9d-b369-546af190c59d" containerID="049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27" exitCode=0
Jul 30 17:24:36.690493 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.690416    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"
Jul 30 17:24:36.690493 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.690438    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp" event={"ID":"a64f3512-5a22-4d9d-b369-546af190c59d","Type":"ContainerDied","Data":"049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27"}
Jul 30 17:24:36.690493 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.690485    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp" event={"ID":"a64f3512-5a22-4d9d-b369-546af190c59d","Type":"ContainerDied","Data":"a8034e307ad0addda52f01aa8004c10798137cbfa4028d388a82e9cce8aa8aa7"}
Jul 30 17:24:36.691801 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.691777    2569 generic.go:356] "Generic (PLEG): container finished" podID="bd3b71d1-515f-47fb-a103-bee705a1c95c" containerID="65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d" exitCode=0
Jul 30 17:24:36.691920 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.691823    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"
Jul 30 17:24:36.691920 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.691849    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl" event={"ID":"bd3b71d1-515f-47fb-a103-bee705a1c95c","Type":"ContainerDied","Data":"65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d"}
Jul 30 17:24:36.691920 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.691886    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl" event={"ID":"bd3b71d1-515f-47fb-a103-bee705a1c95c","Type":"ContainerDied","Data":"df88c4b48e9ae9373407995100a10c54fd5d2a793897906cf3d52937d5c279b1"}
Jul 30 17:24:36.699164 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.699001    2569 scope.go:117] "RemoveContainer" containerID="cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f"
Jul 30 17:24:36.699426 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:24:36.699393    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f\": container with ID starting with cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f not found: ID does not exist" containerID="cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f"
Jul 30 17:24:36.699509 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.699435    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f"} err="failed to get container status \"cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f\": rpc error: code = NotFound desc = could not find container \"cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f\": container with ID starting with cf4ccf316945062aa25ed577e831d405dcb346624f9ed132e0af3ed39f39702f not found: ID does not exist"
Jul 30 17:24:36.699509 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.699456    2569 scope.go:117] "RemoveContainer" containerID="049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27"
Jul 30 17:24:36.707529 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.707508    2569 scope.go:117] "RemoveContainer" containerID="049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27"
Jul 30 17:24:36.707775 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:24:36.707757    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27\": container with ID starting with 049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27 not found: ID does not exist" containerID="049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27"
Jul 30 17:24:36.707838 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.707779    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27"} err="failed to get container status \"049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27\": rpc error: code = NotFound desc = could not find container \"049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27\": container with ID starting with 049c4ed3c2966361fe42350e2c4b642cc84a6148de314165edfdb1febcf60c27 not found: ID does not exist"
Jul 30 17:24:36.707838 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.707805    2569 scope.go:117] "RemoveContainer" containerID="65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d"
Jul 30 17:24:36.714773 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.714757    2569 scope.go:117] "RemoveContainer" containerID="65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d"
Jul 30 17:24:36.715010 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:24:36.714993    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d\": container with ID starting with 65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d not found: ID does not exist" containerID="65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d"
Jul 30 17:24:36.715055 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.715017    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d"} err="failed to get container status \"65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d\": rpc error: code = NotFound desc = could not find container \"65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d\": container with ID starting with 65500c2dfcc88d2000816d90f301d06c3cf6496cce65e71bc10066aef455d43d not found: ID does not exist"
Jul 30 17:24:36.719634 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.719592    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"]
Jul 30 17:24:36.727419 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.727393    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9f05e7-58d6b4f4b7-q2jxr"]
Jul 30 17:24:36.755384 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.755359    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"]
Jul 30 17:24:36.756935 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.756900    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9f05e7-599d498dcd-flzzp"]
Jul 30 17:24:36.768797 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.768777    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"]
Jul 30 17:24:36.780287 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:36.780262    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9f05e7-557f7778f6-8hpvl"]
Jul 30 17:24:37.000574 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:37.000502    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d12a5bb-2f25-4f35-86f4-02cf36216049" path="/var/lib/kubelet/pods/0d12a5bb-2f25-4f35-86f4-02cf36216049/volumes"
Jul 30 17:24:37.000883 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:37.000869    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a64f3512-5a22-4d9d-b369-546af190c59d" path="/var/lib/kubelet/pods/a64f3512-5a22-4d9d-b369-546af190c59d/volumes"
Jul 30 17:24:37.001176 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:37.001164    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd3b71d1-515f-47fb-a103-bee705a1c95c" path="/var/lib/kubelet/pods/bd3b71d1-515f-47fb-a103-bee705a1c95c/volumes"
Jul 30 17:24:53.563558 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.563524    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"]
Jul 30 17:24:53.564115 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.563746    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5" podUID="b1e646fc-d2ea-406d-8286-7f66a12c4015" containerName="maas-api" containerID="cri-o://3fe10b320bd26d86b3d987348a4792536fa06743650888f8369b9e130575db40" gracePeriod=30
Jul 30 17:24:53.634815 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.634785    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"]
Jul 30 17:24:53.635216 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.635021    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m" podUID="18e6e55e-9a1b-4e24-bfd9-13ba9665cd56" containerName="payload-pre-processing" containerID="cri-o://72c9bf3ee23ebb775bca7c0c6869a7bf98651ded482172cbcb7b33e684db0e12" gracePeriod=30
Jul 30 17:24:53.780181 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.780127    2569 generic.go:356] "Generic (PLEG): container finished" podID="18e6e55e-9a1b-4e24-bfd9-13ba9665cd56" containerID="72c9bf3ee23ebb775bca7c0c6869a7bf98651ded482172cbcb7b33e684db0e12" exitCode=0
Jul 30 17:24:53.780373 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.780201    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m" event={"ID":"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56","Type":"ContainerDied","Data":"72c9bf3ee23ebb775bca7c0c6869a7bf98651ded482172cbcb7b33e684db0e12"}
Jul 30 17:24:53.783807 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.783776    2569 generic.go:356] "Generic (PLEG): container finished" podID="b1e646fc-d2ea-406d-8286-7f66a12c4015" containerID="3fe10b320bd26d86b3d987348a4792536fa06743650888f8369b9e130575db40" exitCode=0
Jul 30 17:24:53.784018 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.783829    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5" event={"ID":"b1e646fc-d2ea-406d-8286-7f66a12c4015","Type":"ContainerDied","Data":"3fe10b320bd26d86b3d987348a4792536fa06743650888f8369b9e130575db40"}
Jul 30 17:24:53.888322 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.888299    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:24:53.911832 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.911806    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:24:53.993340 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.993308    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1e646fc-d2ea-406d-8286-7f66a12c4015-maas-api-tls\") pod \"b1e646fc-d2ea-406d-8286-7f66a12c4015\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") "
Jul 30 17:24:53.993496 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.993433    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pkf9\" (UniqueName: \"kubernetes.io/projected/b1e646fc-d2ea-406d-8286-7f66a12c4015-kube-api-access-4pkf9\") pod \"b1e646fc-d2ea-406d-8286-7f66a12c4015\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") "
Jul 30 17:24:53.993496 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.993460    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2dvv\" (UniqueName: \"kubernetes.io/projected/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-kube-api-access-j2dvv\") pod \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\" (UID: \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\") "
Jul 30 17:24:53.993496 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.993481    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1e646fc-d2ea-406d-8286-7f66a12c4015-openshift-service-ca\") pod \"b1e646fc-d2ea-406d-8286-7f66a12c4015\" (UID: \"b1e646fc-d2ea-406d-8286-7f66a12c4015\") "
Jul 30 17:24:53.993657 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.993501    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-plugins-config-volume\") pod \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\" (UID: \"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56\") "
Jul 30 17:24:53.993868 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.993840    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1e646fc-d2ea-406d-8286-7f66a12c4015-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1e646fc-d2ea-406d-8286-7f66a12c4015" (UID: "b1e646fc-d2ea-406d-8286-7f66a12c4015"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:24:53.993955 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.993858    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18e6e55e-9a1b-4e24-bfd9-13ba9665cd56" (UID: "18e6e55e-9a1b-4e24-bfd9-13ba9665cd56"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:24:53.995464 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.995433    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1e646fc-d2ea-406d-8286-7f66a12c4015-kube-api-access-4pkf9" (OuterVolumeSpecName: "kube-api-access-4pkf9") pod "b1e646fc-d2ea-406d-8286-7f66a12c4015" (UID: "b1e646fc-d2ea-406d-8286-7f66a12c4015"). InnerVolumeSpecName "kube-api-access-4pkf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:24:53.995464 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.995437    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1e646fc-d2ea-406d-8286-7f66a12c4015-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1e646fc-d2ea-406d-8286-7f66a12c4015" (UID: "b1e646fc-d2ea-406d-8286-7f66a12c4015"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:24:53.995599 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:53.995469    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-kube-api-access-j2dvv" (OuterVolumeSpecName: "kube-api-access-j2dvv") pod "18e6e55e-9a1b-4e24-bfd9-13ba9665cd56" (UID: "18e6e55e-9a1b-4e24-bfd9-13ba9665cd56"). InnerVolumeSpecName "kube-api-access-j2dvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:24:54.094772 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.094679    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2dvv\" (UniqueName: \"kubernetes.io/projected/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-kube-api-access-j2dvv\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:54.094772 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.094709    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1e646fc-d2ea-406d-8286-7f66a12c4015-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:54.094772 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.094722    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:54.094772 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.094735    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1e646fc-d2ea-406d-8286-7f66a12c4015-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:54.094772 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.094748    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pkf9\" (UniqueName: \"kubernetes.io/projected/b1e646fc-d2ea-406d-8286-7f66a12c4015-kube-api-access-4pkf9\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:24:54.789221 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.789190    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"
Jul 30 17:24:54.789725 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.789195    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m" event={"ID":"18e6e55e-9a1b-4e24-bfd9-13ba9665cd56","Type":"ContainerDied","Data":"dae41ff6d963e3022ad6e5cd4d9d776d02449167cd7e262932044be2c10e3f42"}
Jul 30 17:24:54.789725 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.789333    2569 scope.go:117] "RemoveContainer" containerID="72c9bf3ee23ebb775bca7c0c6869a7bf98651ded482172cbcb7b33e684db0e12"
Jul 30 17:24:54.790705 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.790587    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5" event={"ID":"b1e646fc-d2ea-406d-8286-7f66a12c4015","Type":"ContainerDied","Data":"4b93224576b8fc2fdfd1213386a8a520294346c2b44311ea6ed484f55f6511e8"}
Jul 30 17:24:54.790705 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.790635    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"
Jul 30 17:24:54.799088 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.799073    2569 scope.go:117] "RemoveContainer" containerID="3fe10b320bd26d86b3d987348a4792536fa06743650888f8369b9e130575db40"
Jul 30 17:24:54.812125 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.812105    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"]
Jul 30 17:24:54.813681 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.813663    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c7e9f-6bc4495d7b-gc55m"]
Jul 30 17:24:54.824617 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.824586    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"]
Jul 30 17:24:54.828655 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:54.828592    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4c7e9f-6df9c77654-hgqr5"]
Jul 30 17:24:55.000219 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:55.000188    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e6e55e-9a1b-4e24-bfd9-13ba9665cd56" path="/var/lib/kubelet/pods/18e6e55e-9a1b-4e24-bfd9-13ba9665cd56/volumes"
Jul 30 17:24:55.000597 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:24:55.000584    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1e646fc-d2ea-406d-8286-7f66a12c4015" path="/var/lib/kubelet/pods/b1e646fc-d2ea-406d-8286-7f66a12c4015/volumes"
Jul 30 17:25:05.841708 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.841667    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"]
Jul 30 17:25:05.842091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842017    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d12a5bb-2f25-4f35-86f4-02cf36216049" containerName="maas-api"
Jul 30 17:25:05.842091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842032    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d12a5bb-2f25-4f35-86f4-02cf36216049" containerName="maas-api"
Jul 30 17:25:05.842091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842050    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e6e55e-9a1b-4e24-bfd9-13ba9665cd56" containerName="payload-pre-processing"
Jul 30 17:25:05.842091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842056    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e6e55e-9a1b-4e24-bfd9-13ba9665cd56" containerName="payload-pre-processing"
Jul 30 17:25:05.842091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842065    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a64f3512-5a22-4d9d-b369-546af190c59d" containerName="payload-processing"
Jul 30 17:25:05.842091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842071    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a64f3512-5a22-4d9d-b369-546af190c59d" containerName="payload-processing"
Jul 30 17:25:05.842091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842084    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1e646fc-d2ea-406d-8286-7f66a12c4015" containerName="maas-api"
Jul 30 17:25:05.842091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842089    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1e646fc-d2ea-406d-8286-7f66a12c4015" containerName="maas-api"
Jul 30 17:25:05.842364 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842101    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3b71d1-515f-47fb-a103-bee705a1c95c" containerName="payload-pre-processing"
Jul 30 17:25:05.842364 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842110    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3b71d1-515f-47fb-a103-bee705a1c95c" containerName="payload-pre-processing"
Jul 30 17:25:05.842364 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842164    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e6e55e-9a1b-4e24-bfd9-13ba9665cd56" containerName="payload-pre-processing"
Jul 30 17:25:05.842364 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842171    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d12a5bb-2f25-4f35-86f4-02cf36216049" containerName="maas-api"
Jul 30 17:25:05.842364 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842178    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3b71d1-515f-47fb-a103-bee705a1c95c" containerName="payload-pre-processing"
Jul 30 17:25:05.842364 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842188    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a64f3512-5a22-4d9d-b369-546af190c59d" containerName="payload-processing"
Jul 30 17:25:05.842364 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.842196    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1e646fc-d2ea-406d-8286-7f66a12c4015" containerName="maas-api"
Jul 30 17:25:05.863037 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.863008    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"]
Jul 30 17:25:05.863171 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.863118    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:05.866489 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.866469    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4dfe39-serving-cert\""
Jul 30 17:25:05.993035 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.993005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c9vf\" (UniqueName: \"kubernetes.io/projected/fb2d0674-9231-4105-b4d0-84488fddb497-kube-api-access-7c9vf\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:05.993180 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.993050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2d0674-9231-4105-b4d0-84488fddb497-maas-api-tls\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:05.993180 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:05.993139    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2d0674-9231-4105-b4d0-84488fddb497-openshift-service-ca\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:06.094033 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.093958    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c9vf\" (UniqueName: \"kubernetes.io/projected/fb2d0674-9231-4105-b4d0-84488fddb497-kube-api-access-7c9vf\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:06.094033 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.093999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2d0674-9231-4105-b4d0-84488fddb497-maas-api-tls\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:06.094224 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.094067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2d0674-9231-4105-b4d0-84488fddb497-openshift-service-ca\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:06.094224 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:25:06.094167    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-serving-cert: secret "maas-api-e2e-api-a-4dfe39-serving-cert" not found
Jul 30 17:25:06.094224 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:25:06.094225    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2d0674-9231-4105-b4d0-84488fddb497-maas-api-tls podName:fb2d0674-9231-4105-b4d0-84488fddb497 nodeName:}" failed. No retries permitted until 2026-07-30 17:25:06.594208145 +0000 UTC m=+2934.196635753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fb2d0674-9231-4105-b4d0-84488fddb497-maas-api-tls") pod "maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg" (UID: "fb2d0674-9231-4105-b4d0-84488fddb497") : secret "maas-api-e2e-api-a-4dfe39-serving-cert" not found
Jul 30 17:25:06.094719 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.094699    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2d0674-9231-4105-b4d0-84488fddb497-openshift-service-ca\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:06.108669 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.108648    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c9vf\" (UniqueName: \"kubernetes.io/projected/fb2d0674-9231-4105-b4d0-84488fddb497-kube-api-access-7c9vf\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:06.598445 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.598413    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2d0674-9231-4105-b4d0-84488fddb497-maas-api-tls\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:06.600937 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.600908    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2d0674-9231-4105-b4d0-84488fddb497-maas-api-tls\") pod \"maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:06.772880 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.772849    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:06.802805 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.802776    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"]
Jul 30 17:25:06.805264 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.805242    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:06.810178 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.810158    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4dfe39\""
Jul 30 17:25:06.817344 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.817314    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"]
Jul 30 17:25:06.901174 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.901141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef696fd0-7d11-46b3-8583-b565d2b30073-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22\" (UID: \"ef696fd0-7d11-46b3-8583-b565d2b30073\") " pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:06.901593 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.901188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72hjg\" (UniqueName: \"kubernetes.io/projected/ef696fd0-7d11-46b3-8583-b565d2b30073-kube-api-access-72hjg\") pod \"payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22\" (UID: \"ef696fd0-7d11-46b3-8583-b565d2b30073\") " pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:06.916018 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.915991    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"]
Jul 30 17:25:06.920490 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:25:06.920466    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2d0674_9231_4105_b4d0_84488fddb497.slice/crio-b89c86495c5185557f8eb5924f180fa4409f28a413529e65ac65f703ccf22446 WatchSource:0}: Error finding container b89c86495c5185557f8eb5924f180fa4409f28a413529e65ac65f703ccf22446: Status 404 returned error can't find the container with id b89c86495c5185557f8eb5924f180fa4409f28a413529e65ac65f703ccf22446
Jul 30 17:25:06.921961 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:06.921943    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:25:07.002070 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.002041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef696fd0-7d11-46b3-8583-b565d2b30073-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22\" (UID: \"ef696fd0-7d11-46b3-8583-b565d2b30073\") " pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:07.002205 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.002083    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72hjg\" (UniqueName: \"kubernetes.io/projected/ef696fd0-7d11-46b3-8583-b565d2b30073-kube-api-access-72hjg\") pod \"payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22\" (UID: \"ef696fd0-7d11-46b3-8583-b565d2b30073\") " pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:07.002917 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.002893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef696fd0-7d11-46b3-8583-b565d2b30073-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22\" (UID: \"ef696fd0-7d11-46b3-8583-b565d2b30073\") " pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:07.011338 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.011320    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72hjg\" (UniqueName: \"kubernetes.io/projected/ef696fd0-7d11-46b3-8583-b565d2b30073-kube-api-access-72hjg\") pod \"payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22\" (UID: \"ef696fd0-7d11-46b3-8583-b565d2b30073\") " pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:07.116883 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.116832    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:07.254474 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.254450    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"]
Jul 30 17:25:07.256898 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:25:07.256871    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef696fd0_7d11_46b3_8583_b565d2b30073.slice/crio-604f95beca442adbba6bfde3ef54137648fce39e84ea993165f53fac7e447f7c WatchSource:0}: Error finding container 604f95beca442adbba6bfde3ef54137648fce39e84ea993165f53fac7e447f7c: Status 404 returned error can't find the container with id 604f95beca442adbba6bfde3ef54137648fce39e84ea993165f53fac7e447f7c
Jul 30 17:25:07.856382 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.856293    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22" event={"ID":"ef696fd0-7d11-46b3-8583-b565d2b30073","Type":"ContainerStarted","Data":"d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea"}
Jul 30 17:25:07.856382 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.856361    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:07.856632 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.856389    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22" event={"ID":"ef696fd0-7d11-46b3-8583-b565d2b30073","Type":"ContainerStarted","Data":"604f95beca442adbba6bfde3ef54137648fce39e84ea993165f53fac7e447f7c"}
Jul 30 17:25:07.857862 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.857836    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg" event={"ID":"fb2d0674-9231-4105-b4d0-84488fddb497","Type":"ContainerStarted","Data":"f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750"}
Jul 30 17:25:07.857960 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.857876    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg" event={"ID":"fb2d0674-9231-4105-b4d0-84488fddb497","Type":"ContainerStarted","Data":"b89c86495c5185557f8eb5924f180fa4409f28a413529e65ac65f703ccf22446"}
Jul 30 17:25:07.858004 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.857964    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:07.898016 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.897976    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22" podStartSLOduration=1.89796359 podStartE2EDuration="1.89796359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:25:07.897193527 +0000 UTC m=+2935.499621165" watchObservedRunningTime="2026-07-30 17:25:07.89796359 +0000 UTC m=+2935.500391213"
Jul 30 17:25:07.917561 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:07.917522    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg" podStartSLOduration=2.391719445 podStartE2EDuration="2.917509306s" totalImagesPullingTime="525.789861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:25:07.916076471 +0000 UTC m=+2935.518504098" watchObservedRunningTime="2026-07-30 17:25:07.917509306 +0000 UTC m=+2935.519936941"
Jul 30 17:25:13.868141 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:13.868119    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:25:18.864599 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:18.864572    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:25:34.891594 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:34.891521    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"]
Jul 30 17:25:34.895108 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:34.895090    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:34.901995 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:34.901698    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b9fcce-serving-cert\""
Jul 30 17:25:34.908125 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:34.908099    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"]
Jul 30 17:25:35.026721 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.026696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a427ae-d72b-4e7a-b240-af348873bf11-openshift-service-ca\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.026862 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.026757    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a427ae-d72b-4e7a-b240-af348873bf11-maas-api-tls\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.026862 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.026812    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-758rl\" (UniqueName: \"kubernetes.io/projected/19a427ae-d72b-4e7a-b240-af348873bf11-kube-api-access-758rl\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.127436 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.127404    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-758rl\" (UniqueName: \"kubernetes.io/projected/19a427ae-d72b-4e7a-b240-af348873bf11-kube-api-access-758rl\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.127566 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.127505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a427ae-d72b-4e7a-b240-af348873bf11-openshift-service-ca\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.127740 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.127580    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a427ae-d72b-4e7a-b240-af348873bf11-maas-api-tls\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.127740 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:25:35.127724    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-serving-cert: secret "maas-api-e2e-api-b-b9fcce-serving-cert" not found
Jul 30 17:25:35.127867 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:25:35.127792    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a427ae-d72b-4e7a-b240-af348873bf11-maas-api-tls podName:19a427ae-d72b-4e7a-b240-af348873bf11 nodeName:}" failed. No retries permitted until 2026-07-30 17:25:35.627771762 +0000 UTC m=+2963.230199369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/19a427ae-d72b-4e7a-b240-af348873bf11-maas-api-tls") pod "maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6" (UID: "19a427ae-d72b-4e7a-b240-af348873bf11") : secret "maas-api-e2e-api-b-b9fcce-serving-cert" not found
Jul 30 17:25:35.128139 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.128119    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a427ae-d72b-4e7a-b240-af348873bf11-openshift-service-ca\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.138695 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.138675    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-758rl\" (UniqueName: \"kubernetes.io/projected/19a427ae-d72b-4e7a-b240-af348873bf11-kube-api-access-758rl\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.631984 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.631949    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a427ae-d72b-4e7a-b240-af348873bf11-maas-api-tls\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.634514 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.634486    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a427ae-d72b-4e7a-b240-af348873bf11-maas-api-tls\") pod \"maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.805735 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.805706    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:35.938664 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.935697    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"]
Jul 30 17:25:35.939342 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:25:35.939316    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a427ae_d72b_4e7a_b240_af348873bf11.slice/crio-e8ae62a5011aec6a9c4099722643b7478bfea9b2e195dc1cd8fd7b6d554f0cc3 WatchSource:0}: Error finding container e8ae62a5011aec6a9c4099722643b7478bfea9b2e195dc1cd8fd7b6d554f0cc3: Status 404 returned error can't find the container with id e8ae62a5011aec6a9c4099722643b7478bfea9b2e195dc1cd8fd7b6d554f0cc3
Jul 30 17:25:35.982129 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:35.982106    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6" event={"ID":"19a427ae-d72b-4e7a-b240-af348873bf11","Type":"ContainerStarted","Data":"e8ae62a5011aec6a9c4099722643b7478bfea9b2e195dc1cd8fd7b6d554f0cc3"}
Jul 30 17:25:36.988392 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:36.988357    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6" event={"ID":"19a427ae-d72b-4e7a-b240-af348873bf11","Type":"ContainerStarted","Data":"d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b"}
Jul 30 17:25:36.988860 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:36.988486    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:25:37.006884 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:37.006836    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6" podStartSLOduration=2.590678554 podStartE2EDuration="3.006823412s" totalImagesPullingTime="416.144858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:25:37.006750902 +0000 UTC m=+2964.609178532" watchObservedRunningTime="2026-07-30 17:25:37.006823412 +0000 UTC m=+2964.609251075"
Jul 30 17:25:43.000211 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:25:43.000183    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:26:17.511821 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.511253    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"]
Jul 30 17:26:17.512330 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.512039    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg" podUID="fb2d0674-9231-4105-b4d0-84488fddb497" containerName="maas-api" containerID="cri-o://f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750" gracePeriod=30
Jul 30 17:26:17.574746 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.574636    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"]
Jul 30 17:26:17.575208 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.575160    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22" podUID="ef696fd0-7d11-46b3-8583-b565d2b30073" containerName="payload-processing" containerID="cri-o://d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea" gracePeriod=30
Jul 30 17:26:17.828942 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.828916    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:26:17.859850 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.859823    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:26:17.861899 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.861874    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2d0674-9231-4105-b4d0-84488fddb497-openshift-service-ca\") pod \"fb2d0674-9231-4105-b4d0-84488fddb497\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") "
Jul 30 17:26:17.862025 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.861966    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2d0674-9231-4105-b4d0-84488fddb497-maas-api-tls\") pod \"fb2d0674-9231-4105-b4d0-84488fddb497\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") "
Jul 30 17:26:17.862076 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.862065    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7c9vf\" (UniqueName: \"kubernetes.io/projected/fb2d0674-9231-4105-b4d0-84488fddb497-kube-api-access-7c9vf\") pod \"fb2d0674-9231-4105-b4d0-84488fddb497\" (UID: \"fb2d0674-9231-4105-b4d0-84488fddb497\") "
Jul 30 17:26:17.862264 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.862241    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb2d0674-9231-4105-b4d0-84488fddb497-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb2d0674-9231-4105-b4d0-84488fddb497" (UID: "fb2d0674-9231-4105-b4d0-84488fddb497"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:26:17.862357 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.862344    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2d0674-9231-4105-b4d0-84488fddb497-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:26:17.864078 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.864053    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb2d0674-9231-4105-b4d0-84488fddb497-kube-api-access-7c9vf" (OuterVolumeSpecName: "kube-api-access-7c9vf") pod "fb2d0674-9231-4105-b4d0-84488fddb497" (UID: "fb2d0674-9231-4105-b4d0-84488fddb497"). InnerVolumeSpecName "kube-api-access-7c9vf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:26:17.864166 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.864056    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb2d0674-9231-4105-b4d0-84488fddb497-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb2d0674-9231-4105-b4d0-84488fddb497" (UID: "fb2d0674-9231-4105-b4d0-84488fddb497"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:26:17.962855 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.962823    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72hjg\" (UniqueName: \"kubernetes.io/projected/ef696fd0-7d11-46b3-8583-b565d2b30073-kube-api-access-72hjg\") pod \"ef696fd0-7d11-46b3-8583-b565d2b30073\" (UID: \"ef696fd0-7d11-46b3-8583-b565d2b30073\") "
Jul 30 17:26:17.963047 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.962866    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef696fd0-7d11-46b3-8583-b565d2b30073-plugins-config-volume\") pod \"ef696fd0-7d11-46b3-8583-b565d2b30073\" (UID: \"ef696fd0-7d11-46b3-8583-b565d2b30073\") "
Jul 30 17:26:17.963158 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.963111    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb2d0674-9231-4105-b4d0-84488fddb497-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:26:17.963158 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.963135    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7c9vf\" (UniqueName: \"kubernetes.io/projected/fb2d0674-9231-4105-b4d0-84488fddb497-kube-api-access-7c9vf\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:26:17.963283 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.963247    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef696fd0-7d11-46b3-8583-b565d2b30073-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef696fd0-7d11-46b3-8583-b565d2b30073" (UID: "ef696fd0-7d11-46b3-8583-b565d2b30073"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:26:17.964897 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:17.964873    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef696fd0-7d11-46b3-8583-b565d2b30073-kube-api-access-72hjg" (OuterVolumeSpecName: "kube-api-access-72hjg") pod "ef696fd0-7d11-46b3-8583-b565d2b30073" (UID: "ef696fd0-7d11-46b3-8583-b565d2b30073"). InnerVolumeSpecName "kube-api-access-72hjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:26:18.063662 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.063568    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef696fd0-7d11-46b3-8583-b565d2b30073-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:26:18.063662 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.063595    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72hjg\" (UniqueName: \"kubernetes.io/projected/ef696fd0-7d11-46b3-8583-b565d2b30073-kube-api-access-72hjg\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:26:18.174118 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.174085    2569 generic.go:356] "Generic (PLEG): container finished" podID="fb2d0674-9231-4105-b4d0-84488fddb497" containerID="f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750" exitCode=0
Jul 30 17:26:18.174312 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.174132    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"
Jul 30 17:26:18.174312 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.174171    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg" event={"ID":"fb2d0674-9231-4105-b4d0-84488fddb497","Type":"ContainerDied","Data":"f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750"}
Jul 30 17:26:18.174312 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.174226    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg" event={"ID":"fb2d0674-9231-4105-b4d0-84488fddb497","Type":"ContainerDied","Data":"b89c86495c5185557f8eb5924f180fa4409f28a413529e65ac65f703ccf22446"}
Jul 30 17:26:18.174312 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.174255    2569 scope.go:117] "RemoveContainer" containerID="f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750"
Jul 30 17:26:18.175522 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.175502    2569 generic.go:356] "Generic (PLEG): container finished" podID="ef696fd0-7d11-46b3-8583-b565d2b30073" containerID="d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea" exitCode=0
Jul 30 17:26:18.175629 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.175576    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22" event={"ID":"ef696fd0-7d11-46b3-8583-b565d2b30073","Type":"ContainerDied","Data":"d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea"}
Jul 30 17:26:18.175629 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.175592    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"
Jul 30 17:26:18.175727 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.175641    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22" event={"ID":"ef696fd0-7d11-46b3-8583-b565d2b30073","Type":"ContainerDied","Data":"604f95beca442adbba6bfde3ef54137648fce39e84ea993165f53fac7e447f7c"}
Jul 30 17:26:18.185089 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.185035    2569 scope.go:117] "RemoveContainer" containerID="f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750"
Jul 30 17:26:18.185434 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:26:18.185316    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750\": container with ID starting with f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750 not found: ID does not exist" containerID="f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750"
Jul 30 17:26:18.185434 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.185341    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750"} err="failed to get container status \"f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750\": rpc error: code = NotFound desc = could not find container \"f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750\": container with ID starting with f281b3d45360cfeda7ca7597bfc1dd16b008c2f3d3722203ce6ccd15e2735750 not found: ID does not exist"
Jul 30 17:26:18.185434 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.185358    2569 scope.go:117] "RemoveContainer" containerID="d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea"
Jul 30 17:26:18.193077 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.193056    2569 scope.go:117] "RemoveContainer" containerID="d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea"
Jul 30 17:26:18.193333 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:26:18.193313    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea\": container with ID starting with d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea not found: ID does not exist" containerID="d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea"
Jul 30 17:26:18.193377 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.193344    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea"} err="failed to get container status \"d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea\": rpc error: code = NotFound desc = could not find container \"d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea\": container with ID starting with d950bba6f7793d0c6eff7f1b0d56aabbc8d643c4ea63999919b4b08c8dde68ea not found: ID does not exist"
Jul 30 17:26:18.197565 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.197538    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"]
Jul 30 17:26:18.200765 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.200743    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4dfe39-65cfb4d74c-rgvpg"]
Jul 30 17:26:18.211764 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.211735    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"]
Jul 30 17:26:18.215372 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:18.215345    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4dfe39-68dc47d698-wzd22"]
Jul 30 17:26:19.000797 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:19.000759    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef696fd0-7d11-46b3-8583-b565d2b30073" path="/var/lib/kubelet/pods/ef696fd0-7d11-46b3-8583-b565d2b30073/volumes"
Jul 30 17:26:19.001165 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:19.001139    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb2d0674-9231-4105-b4d0-84488fddb497" path="/var/lib/kubelet/pods/fb2d0674-9231-4105-b4d0-84488fddb497/volumes"
Jul 30 17:26:40.305718 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.305685    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"]
Jul 30 17:26:40.306220 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.305906    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6" podUID="19a427ae-d72b-4e7a-b240-af348873bf11" containerName="maas-api" containerID="cri-o://d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b" gracePeriod=30
Jul 30 17:26:40.548997 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.548972    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:26:40.664420 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.664390    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a427ae-d72b-4e7a-b240-af348873bf11-openshift-service-ca\") pod \"19a427ae-d72b-4e7a-b240-af348873bf11\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") "
Jul 30 17:26:40.664581 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.664426    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a427ae-d72b-4e7a-b240-af348873bf11-maas-api-tls\") pod \"19a427ae-d72b-4e7a-b240-af348873bf11\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") "
Jul 30 17:26:40.664581 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.664482    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-758rl\" (UniqueName: \"kubernetes.io/projected/19a427ae-d72b-4e7a-b240-af348873bf11-kube-api-access-758rl\") pod \"19a427ae-d72b-4e7a-b240-af348873bf11\" (UID: \"19a427ae-d72b-4e7a-b240-af348873bf11\") "
Jul 30 17:26:40.664816 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.664781    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a427ae-d72b-4e7a-b240-af348873bf11-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "19a427ae-d72b-4e7a-b240-af348873bf11" (UID: "19a427ae-d72b-4e7a-b240-af348873bf11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:26:40.666456 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.666428    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a427ae-d72b-4e7a-b240-af348873bf11-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "19a427ae-d72b-4e7a-b240-af348873bf11" (UID: "19a427ae-d72b-4e7a-b240-af348873bf11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:26:40.666531 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.666494    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19a427ae-d72b-4e7a-b240-af348873bf11-kube-api-access-758rl" (OuterVolumeSpecName: "kube-api-access-758rl") pod "19a427ae-d72b-4e7a-b240-af348873bf11" (UID: "19a427ae-d72b-4e7a-b240-af348873bf11"). InnerVolumeSpecName "kube-api-access-758rl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:26:40.765077 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.765037    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a427ae-d72b-4e7a-b240-af348873bf11-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:26:40.765077 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.765072    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19a427ae-d72b-4e7a-b240-af348873bf11-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:26:40.765077 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:40.765081    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-758rl\" (UniqueName: \"kubernetes.io/projected/19a427ae-d72b-4e7a-b240-af348873bf11-kube-api-access-758rl\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:26:41.282914 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:41.282818    2569 generic.go:356] "Generic (PLEG): container finished" podID="19a427ae-d72b-4e7a-b240-af348873bf11" containerID="d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b" exitCode=0
Jul 30 17:26:41.282914 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:41.282866    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"
Jul 30 17:26:41.282914 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:41.282902    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6" event={"ID":"19a427ae-d72b-4e7a-b240-af348873bf11","Type":"ContainerDied","Data":"d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b"}
Jul 30 17:26:41.283200 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:41.282956    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6" event={"ID":"19a427ae-d72b-4e7a-b240-af348873bf11","Type":"ContainerDied","Data":"e8ae62a5011aec6a9c4099722643b7478bfea9b2e195dc1cd8fd7b6d554f0cc3"}
Jul 30 17:26:41.283200 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:41.282979    2569 scope.go:117] "RemoveContainer" containerID="d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b"
Jul 30 17:26:41.291165 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:41.291145    2569 scope.go:117] "RemoveContainer" containerID="d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b"
Jul 30 17:26:41.291528 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:26:41.291456    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b\": container with ID starting with d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b not found: ID does not exist" containerID="d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b"
Jul 30 17:26:41.291528 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:41.291495    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b"} err="failed to get container status \"d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b\": rpc error: code = NotFound desc = could not find container \"d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b\": container with ID starting with d6c6bd583e3e5d4da9dc7d7054a7584f322de5de55816de8880ca6bcb3dfd94b not found: ID does not exist"
Jul 30 17:26:41.330179 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:41.330149    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"]
Jul 30 17:26:41.340880 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:41.340855    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b9fcce-6d7d8f999-d6qd6"]
Jul 30 17:26:43.000617 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:43.000576    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a427ae-d72b-4e7a-b240-af348873bf11" path="/var/lib/kubelet/pods/19a427ae-d72b-4e7a-b240-af348873bf11/volumes"
Jul 30 17:26:53.605703 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.605669    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"]
Jul 30 17:26:53.606155 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.606050    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb2d0674-9231-4105-b4d0-84488fddb497" containerName="maas-api"
Jul 30 17:26:53.606155 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.606062    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb2d0674-9231-4105-b4d0-84488fddb497" containerName="maas-api"
Jul 30 17:26:53.606155 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.606073    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef696fd0-7d11-46b3-8583-b565d2b30073" containerName="payload-processing"
Jul 30 17:26:53.606155 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.606078    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef696fd0-7d11-46b3-8583-b565d2b30073" containerName="payload-processing"
Jul 30 17:26:53.606155 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.606087    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a427ae-d72b-4e7a-b240-af348873bf11" containerName="maas-api"
Jul 30 17:26:53.606155 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.606092    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a427ae-d72b-4e7a-b240-af348873bf11" containerName="maas-api"
Jul 30 17:26:53.606155 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.606147    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef696fd0-7d11-46b3-8583-b565d2b30073" containerName="payload-processing"
Jul 30 17:26:53.606155 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.606157    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a427ae-d72b-4e7a-b240-af348873bf11" containerName="maas-api"
Jul 30 17:26:53.606406 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.606165    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb2d0674-9231-4105-b4d0-84488fddb497" containerName="maas-api"
Jul 30 17:26:53.608931 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.608916    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:53.611966 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.611945    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-f45123-serving-cert\""
Jul 30 17:26:53.634695 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.634671    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"]
Jul 30 17:26:53.669348 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.669321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a211841-3798-4dd0-8ce9-bac2ab04c310-maas-api-tls\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:53.669494 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.669474    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsnlc\" (UniqueName: \"kubernetes.io/projected/3a211841-3798-4dd0-8ce9-bac2ab04c310-kube-api-access-vsnlc\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:53.669572 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.669558    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a211841-3798-4dd0-8ce9-bac2ab04c310-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:53.769941 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.769913    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsnlc\" (UniqueName: \"kubernetes.io/projected/3a211841-3798-4dd0-8ce9-bac2ab04c310-kube-api-access-vsnlc\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:53.770059 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.769964    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a211841-3798-4dd0-8ce9-bac2ab04c310-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:53.770059 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.770030    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a211841-3798-4dd0-8ce9-bac2ab04c310-maas-api-tls\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:53.770170 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:26:53.770148    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-serving-cert: secret "maas-api-e2e-shared-a-f45123-serving-cert" not found
Jul 30 17:26:53.770230 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:26:53.770218    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a211841-3798-4dd0-8ce9-bac2ab04c310-maas-api-tls podName:3a211841-3798-4dd0-8ce9-bac2ab04c310 nodeName:}" failed. No retries permitted until 2026-07-30 17:26:54.270196991 +0000 UTC m=+3041.872624609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a211841-3798-4dd0-8ce9-bac2ab04c310-maas-api-tls") pod "maas-api-e2e-shared-a-f45123-74776fb64-fjl65" (UID: "3a211841-3798-4dd0-8ce9-bac2ab04c310") : secret "maas-api-e2e-shared-a-f45123-serving-cert" not found
Jul 30 17:26:53.770537 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.770518    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a211841-3798-4dd0-8ce9-bac2ab04c310-openshift-service-ca\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:53.780770 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:53.780745    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsnlc\" (UniqueName: \"kubernetes.io/projected/3a211841-3798-4dd0-8ce9-bac2ab04c310-kube-api-access-vsnlc\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:54.273270 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.273240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a211841-3798-4dd0-8ce9-bac2ab04c310-maas-api-tls\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:54.275446 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.275416    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a211841-3798-4dd0-8ce9-bac2ab04c310-maas-api-tls\") pod \"maas-api-e2e-shared-a-f45123-74776fb64-fjl65\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:54.507804 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.507771    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"]
Jul 30 17:26:54.511230 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.511208    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:26:54.517816 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.517756    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-f45123\""
Jul 30 17:26:54.518030 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.518010    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:54.518310 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.518285    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-f45123-dockercfg-z5tk9\""
Jul 30 17:26:54.528341 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.528285    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"]
Jul 30 17:26:54.575339 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.575246    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj\" (UID: \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:26:54.575339 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.575308    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb6sw\" (UniqueName: \"kubernetes.io/projected/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-kube-api-access-zb6sw\") pod \"payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj\" (UID: \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:26:54.658525 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.658495    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"]
Jul 30 17:26:54.660534 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:26:54.660507    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a211841_3798_4dd0_8ce9_bac2ab04c310.slice/crio-c19dec5b476e5853f133354e984cd095064d4da16960e335d7f7edaab2522db5 WatchSource:0}: Error finding container c19dec5b476e5853f133354e984cd095064d4da16960e335d7f7edaab2522db5: Status 404 returned error can't find the container with id c19dec5b476e5853f133354e984cd095064d4da16960e335d7f7edaab2522db5
Jul 30 17:26:54.675777 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.675753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj\" (UID: \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:26:54.675865 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.675796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb6sw\" (UniqueName: \"kubernetes.io/projected/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-kube-api-access-zb6sw\") pod \"payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj\" (UID: \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:26:54.676467 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.676451    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj\" (UID: \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:26:54.686331 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.686309    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb6sw\" (UniqueName: \"kubernetes.io/projected/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-kube-api-access-zb6sw\") pod \"payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj\" (UID: \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:26:54.821694 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.821623    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:26:54.959448 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:54.959419    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"]
Jul 30 17:26:54.961489 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:26:54.961463    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa95483_f68f_4bf0_97b5_2ef6cfb7c1ea.slice/crio-0292acc073270db67f27f62242fa7c31cd2552badadc5e653ca6f8195c094c2e WatchSource:0}: Error finding container 0292acc073270db67f27f62242fa7c31cd2552badadc5e653ca6f8195c094c2e: Status 404 returned error can't find the container with id 0292acc073270db67f27f62242fa7c31cd2552badadc5e653ca6f8195c094c2e
Jul 30 17:26:55.346517 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:55.346426    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj" event={"ID":"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea","Type":"ContainerStarted","Data":"50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c"}
Jul 30 17:26:55.346517 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:55.346481    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj" event={"ID":"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea","Type":"ContainerStarted","Data":"0292acc073270db67f27f62242fa7c31cd2552badadc5e653ca6f8195c094c2e"}
Jul 30 17:26:55.346772 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:55.346531    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:26:55.347998 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:55.347973    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65" event={"ID":"3a211841-3798-4dd0-8ce9-bac2ab04c310","Type":"ContainerStarted","Data":"79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84"}
Jul 30 17:26:55.348127 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:55.348015    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65" event={"ID":"3a211841-3798-4dd0-8ce9-bac2ab04c310","Type":"ContainerStarted","Data":"c19dec5b476e5853f133354e984cd095064d4da16960e335d7f7edaab2522db5"}
Jul 30 17:26:55.348127 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:55.348045    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:26:55.375559 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:55.375510    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj" podStartSLOduration=1.375496794 podStartE2EDuration="1.375496794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:26:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:26:55.374159102 +0000 UTC m=+3042.976586741" watchObservedRunningTime="2026-07-30 17:26:55.375496794 +0000 UTC m=+3042.977924419"
Jul 30 17:26:55.405320 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:26:55.405273    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65" podStartSLOduration=2.007752257 podStartE2EDuration="2.405260498s" totalImagesPullingTime="397.508241ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:26:55.403551092 +0000 UTC m=+3043.005978717" watchObservedRunningTime="2026-07-30 17:26:55.405260498 +0000 UTC m=+3043.007688123"
Jul 30 17:27:01.358846 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:01.358815    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:27:06.355561 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:06.355484    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:27:22.707305 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.707275    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"]
Jul 30 17:27:22.712520 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.712498    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:22.715260 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.715235    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-27deb9-serving-cert\""
Jul 30 17:27:22.722999 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.722973    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"]
Jul 30 17:27:22.806093 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.806071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea41a796-3d52-4fc9-9b4b-1643f92a474f-maas-api-tls\") pod \"maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:22.806196 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.806145    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea41a796-3d52-4fc9-9b4b-1643f92a474f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:22.806196 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.806163    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twps8\" (UniqueName: \"kubernetes.io/projected/ea41a796-3d52-4fc9-9b4b-1643f92a474f-kube-api-access-twps8\") pod \"maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:22.906697 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.906673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea41a796-3d52-4fc9-9b4b-1643f92a474f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:22.906793 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.906699    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twps8\" (UniqueName: \"kubernetes.io/projected/ea41a796-3d52-4fc9-9b4b-1643f92a474f-kube-api-access-twps8\") pod \"maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:22.906793 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.906730    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea41a796-3d52-4fc9-9b4b-1643f92a474f-maas-api-tls\") pod \"maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:22.907300 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.907250    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea41a796-3d52-4fc9-9b4b-1643f92a474f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:22.909237 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.909217    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea41a796-3d52-4fc9-9b4b-1643f92a474f-maas-api-tls\") pod \"maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:22.918684 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:22.918661    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twps8\" (UniqueName: \"kubernetes.io/projected/ea41a796-3d52-4fc9-9b4b-1643f92a474f-kube-api-access-twps8\") pod \"maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:23.022903 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:23.022849    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:23.160409 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:23.160383    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"]
Jul 30 17:27:23.162111 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:27:23.162085    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea41a796_3d52_4fc9_9b4b_1643f92a474f.slice/crio-59a35737e1ad8984ed31c67898db883dfaf1c40491a2c6fedcbc3d60229ebb8a WatchSource:0}: Error finding container 59a35737e1ad8984ed31c67898db883dfaf1c40491a2c6fedcbc3d60229ebb8a: Status 404 returned error can't find the container with id 59a35737e1ad8984ed31c67898db883dfaf1c40491a2c6fedcbc3d60229ebb8a
Jul 30 17:27:23.478163 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:23.478131    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf" event={"ID":"ea41a796-3d52-4fc9-9b4b-1643f92a474f","Type":"ContainerStarted","Data":"59a35737e1ad8984ed31c67898db883dfaf1c40491a2c6fedcbc3d60229ebb8a"}
Jul 30 17:27:24.484687 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:24.484648    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf" event={"ID":"ea41a796-3d52-4fc9-9b4b-1643f92a474f","Type":"ContainerStarted","Data":"517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1"}
Jul 30 17:27:24.485077 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:24.484905    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:27:24.503353 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:24.503303    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf" podStartSLOduration=1.910825935 podStartE2EDuration="2.503289916s" totalImagesPullingTime="592.463981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:24.502233802 +0000 UTC m=+3072.104661429" watchObservedRunningTime="2026-07-30 17:27:24.503289916 +0000 UTC m=+3072.105717541"
Jul 30 17:27:30.495250 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:27:30.495210    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:30:00.146795 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:00.146758    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4"]
Jul 30 17:30:00.150388 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:00.150361    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4"
Jul 30 17:30:00.154240 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:00.154215    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:30:00.161886 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:00.161863    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4"
Jul 30 17:30:00.180386 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:00.180361    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4"]
Jul 30 17:30:00.305818 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:00.305781    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4"]
Jul 30 17:30:00.313695 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:30:00.313671    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88ed3121_f6ab_4249_bd48_8be02f38c252.slice/crio-aa27b306a4e6badb44587264632e6b9447e48841769f16275c2285a357923ef2 WatchSource:0}: Error finding container aa27b306a4e6badb44587264632e6b9447e48841769f16275c2285a357923ef2: Status 404 returned error can't find the container with id aa27b306a4e6badb44587264632e6b9447e48841769f16275c2285a357923ef2
Jul 30 17:30:01.217523 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:01.217488    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4" event={"ID":"88ed3121-f6ab-4249-bd48-8be02f38c252","Type":"ContainerStarted","Data":"aa27b306a4e6badb44587264632e6b9447e48841769f16275c2285a357923ef2"}
Jul 30 17:30:03.230202 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:03.230174    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4" event={"ID":"88ed3121-f6ab-4249-bd48-8be02f38c252","Type":"ContainerStarted","Data":"5c3ca93916347fc40f873fa6d220b97cac46bb15d153b66e663c36f76b883181"}
Jul 30 17:30:03.249392 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:03.249349    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4" podStartSLOduration=0.890398287 podStartE2EDuration="3.24933554s" totalImagesPullingTime="2.358937253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:30:03.248308817 +0000 UTC m=+3230.850736442" watchObservedRunningTime="2026-07-30 17:30:03.24933554 +0000 UTC m=+3230.851763188"
Jul 30 17:30:04.236500 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:04.236463    2569 generic.go:356] "Generic (PLEG): container finished" podID="88ed3121-f6ab-4249-bd48-8be02f38c252" containerID="5c3ca93916347fc40f873fa6d220b97cac46bb15d153b66e663c36f76b883181" exitCode=0
Jul 30 17:30:04.236888 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:04.236533    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4" event={"ID":"88ed3121-f6ab-4249-bd48-8be02f38c252","Type":"ContainerDied","Data":"5c3ca93916347fc40f873fa6d220b97cac46bb15d153b66e663c36f76b883181"}
Jul 30 17:30:05.381595 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:05.381573    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4"
Jul 30 17:30:06.249736 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:06.249698    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4"
Jul 30 17:30:06.249736 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:06.249727    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757210-nbqr4" event={"ID":"88ed3121-f6ab-4249-bd48-8be02f38c252","Type":"ContainerDied","Data":"aa27b306a4e6badb44587264632e6b9447e48841769f16275c2285a357923ef2"}
Jul 30 17:30:06.249976 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:30:06.249783    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa27b306a4e6badb44587264632e6b9447e48841769f16275c2285a357923ef2"
Jul 30 17:38:15.601352 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.601304    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7785-140008135662256"]
Jul 30 17:38:15.601835 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.601694    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88ed3121-f6ab-4249-bd48-8be02f38c252" containerName="cleanup"
Jul 30 17:38:15.601835 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.601705    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="88ed3121-f6ab-4249-bd48-8be02f38c252" containerName="cleanup"
Jul 30 17:38:15.601835 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.601769    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="88ed3121-f6ab-4249-bd48-8be02f38c252" containerName="cleanup"
Jul 30 17:38:15.604761 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.604746    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135662256"
Jul 30 17:38:15.607295 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.607276    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4sss8\""
Jul 30 17:38:15.612243 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.611870    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135662256"]
Jul 30 17:38:15.614727 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.614709    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtthg\" (UniqueName: \"kubernetes.io/projected/a3d4d224-d977-4833-a0f9-5e89cf2276a1-kube-api-access-mtthg\") pod \"test-curl-7785-140008135662256\" (UID: \"a3d4d224-d977-4833-a0f9-5e89cf2276a1\") " pod="opendatahub/test-curl-7785-140008135662256"
Jul 30 17:38:15.715783 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.715760    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtthg\" (UniqueName: \"kubernetes.io/projected/a3d4d224-d977-4833-a0f9-5e89cf2276a1-kube-api-access-mtthg\") pod \"test-curl-7785-140008135662256\" (UID: \"a3d4d224-d977-4833-a0f9-5e89cf2276a1\") " pod="opendatahub/test-curl-7785-140008135662256"
Jul 30 17:38:15.725154 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.725135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtthg\" (UniqueName: \"kubernetes.io/projected/a3d4d224-d977-4833-a0f9-5e89cf2276a1-kube-api-access-mtthg\") pod \"test-curl-7785-140008135662256\" (UID: \"a3d4d224-d977-4833-a0f9-5e89cf2276a1\") " pod="opendatahub/test-curl-7785-140008135662256"
Jul 30 17:38:15.914836 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:15.914815    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135662256"
Jul 30 17:38:16.040125 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:16.040100    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135662256"]
Jul 30 17:38:16.042590 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:38:16.042561    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3d4d224_d977_4833_a0f9_5e89cf2276a1.slice/crio-4e3996fc7d6e04fa790f87e97382601b4deb50b4bb98657395df45ad55ccd302 WatchSource:0}: Error finding container 4e3996fc7d6e04fa790f87e97382601b4deb50b4bb98657395df45ad55ccd302: Status 404 returned error can't find the container with id 4e3996fc7d6e04fa790f87e97382601b4deb50b4bb98657395df45ad55ccd302
Jul 30 17:38:16.043859 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:16.043838    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:38:16.498251 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:16.498212    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135662256" event={"ID":"a3d4d224-d977-4833-a0f9-5e89cf2276a1","Type":"ContainerStarted","Data":"4e3996fc7d6e04fa790f87e97382601b4deb50b4bb98657395df45ad55ccd302"}
Jul 30 17:38:17.503929 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:17.503888    2569 generic.go:356] "Generic (PLEG): container finished" podID="a3d4d224-d977-4833-a0f9-5e89cf2276a1" containerID="9f652163706f9849a63fc90ae3dd39f6c48464e8a60d80dea049173bcd8fbb9e" exitCode=0
Jul 30 17:38:17.504354 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:17.503967    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135662256" event={"ID":"a3d4d224-d977-4833-a0f9-5e89cf2276a1","Type":"ContainerDied","Data":"9f652163706f9849a63fc90ae3dd39f6c48464e8a60d80dea049173bcd8fbb9e"}
Jul 30 17:38:18.639467 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:18.639444    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135662256"
Jul 30 17:38:18.676777 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:18.676758    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7785-140008135662256_a3d4d224-d977-4833-a0f9-5e89cf2276a1/test-curl-7785-140008135662256/0.log"
Jul 30 17:38:18.702169 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:18.702144    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7785-140008135662256"]
Jul 30 17:38:18.708200 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:18.708179    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7785-140008135662256"]
Jul 30 17:38:18.737507 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:18.737484    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtthg\" (UniqueName: \"kubernetes.io/projected/a3d4d224-d977-4833-a0f9-5e89cf2276a1-kube-api-access-mtthg\") pod \"a3d4d224-d977-4833-a0f9-5e89cf2276a1\" (UID: \"a3d4d224-d977-4833-a0f9-5e89cf2276a1\") "
Jul 30 17:38:18.739387 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:18.739365    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d4d224-d977-4833-a0f9-5e89cf2276a1-kube-api-access-mtthg" (OuterVolumeSpecName: "kube-api-access-mtthg") pod "a3d4d224-d977-4833-a0f9-5e89cf2276a1" (UID: "a3d4d224-d977-4833-a0f9-5e89cf2276a1"). InnerVolumeSpecName "kube-api-access-mtthg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:18.838643 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:18.838572    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtthg\" (UniqueName: \"kubernetes.io/projected/a3d4d224-d977-4833-a0f9-5e89cf2276a1-kube-api-access-mtthg\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:38:19.000113 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:19.000082    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3d4d224-d977-4833-a0f9-5e89cf2276a1" path="/var/lib/kubelet/pods/a3d4d224-d977-4833-a0f9-5e89cf2276a1/volumes"
Jul 30 17:38:19.513997 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:19.513976    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135662256"
Jul 30 17:38:19.513997 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:19.513977    2569 scope.go:117] "RemoveContainer" containerID="9f652163706f9849a63fc90ae3dd39f6c48464e8a60d80dea049173bcd8fbb9e"
Jul 30 17:38:25.116149 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.116118    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:25.116512 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.116479    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3d4d224-d977-4833-a0f9-5e89cf2276a1" containerName="test-curl-7785-140008135662256"
Jul 30 17:38:25.116512 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.116491    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3d4d224-d977-4833-a0f9-5e89cf2276a1" containerName="test-curl-7785-140008135662256"
Jul 30 17:38:25.116583 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.116551    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3d4d224-d977-4833-a0f9-5e89cf2276a1" containerName="test-curl-7785-140008135662256"
Jul 30 17:38:25.121016 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.121000    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:25.123752 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.123728    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4sss8\""
Jul 30 17:38:25.124415 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.124392    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:25.180861 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.180830    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scrb8\" (UniqueName: \"kubernetes.io/projected/7781c862-045b-4fd8-b01e-dd8dd9ebae5f-kube-api-access-scrb8\") pod \"test-curl-7785-140008135781520\" (UID: \"7781c862-045b-4fd8-b01e-dd8dd9ebae5f\") " pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:25.281437 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.281411    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scrb8\" (UniqueName: \"kubernetes.io/projected/7781c862-045b-4fd8-b01e-dd8dd9ebae5f-kube-api-access-scrb8\") pod \"test-curl-7785-140008135781520\" (UID: \"7781c862-045b-4fd8-b01e-dd8dd9ebae5f\") " pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:25.290969 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.290947    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scrb8\" (UniqueName: \"kubernetes.io/projected/7781c862-045b-4fd8-b01e-dd8dd9ebae5f-kube-api-access-scrb8\") pod \"test-curl-7785-140008135781520\" (UID: \"7781c862-045b-4fd8-b01e-dd8dd9ebae5f\") " pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:25.431717 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.431692    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:25.562502 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:25.562480    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:25.564330 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:38:25.564302    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7781c862_045b_4fd8_b01e_dd8dd9ebae5f.slice/crio-e08d372dc9d75dcd460223f00ef88a981ca8431d73ba966775ee68ddfe329619 WatchSource:0}: Error finding container e08d372dc9d75dcd460223f00ef88a981ca8431d73ba966775ee68ddfe329619: Status 404 returned error can't find the container with id e08d372dc9d75dcd460223f00ef88a981ca8431d73ba966775ee68ddfe329619
Jul 30 17:38:26.548171 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:26.548103    2569 generic.go:356] "Generic (PLEG): container finished" podID="7781c862-045b-4fd8-b01e-dd8dd9ebae5f" containerID="b9c630919271e56fa49fc60ae7780301b7b5162c5b76dce9967e8cf61c7bfc8b" exitCode=0
Jul 30 17:38:26.548482 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:26.548180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135781520" event={"ID":"7781c862-045b-4fd8-b01e-dd8dd9ebae5f","Type":"ContainerDied","Data":"b9c630919271e56fa49fc60ae7780301b7b5162c5b76dce9967e8cf61c7bfc8b"}
Jul 30 17:38:26.548482 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:26.548217    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7785-140008135781520" event={"ID":"7781c862-045b-4fd8-b01e-dd8dd9ebae5f","Type":"ContainerStarted","Data":"e08d372dc9d75dcd460223f00ef88a981ca8431d73ba966775ee68ddfe329619"}
Jul 30 17:38:27.678149 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:27.678125    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:27.701112 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:27.701090    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7785-140008135781520_7781c862-045b-4fd8-b01e-dd8dd9ebae5f/test-curl-7785-140008135781520/0.log"
Jul 30 17:38:27.702290 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:27.702270    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scrb8\" (UniqueName: \"kubernetes.io/projected/7781c862-045b-4fd8-b01e-dd8dd9ebae5f-kube-api-access-scrb8\") pod \"7781c862-045b-4fd8-b01e-dd8dd9ebae5f\" (UID: \"7781c862-045b-4fd8-b01e-dd8dd9ebae5f\") "
Jul 30 17:38:27.704248 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:27.704228    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7781c862-045b-4fd8-b01e-dd8dd9ebae5f-kube-api-access-scrb8" (OuterVolumeSpecName: "kube-api-access-scrb8") pod "7781c862-045b-4fd8-b01e-dd8dd9ebae5f" (UID: "7781c862-045b-4fd8-b01e-dd8dd9ebae5f"). InnerVolumeSpecName "kube-api-access-scrb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:27.728048 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:27.728023    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:27.731706 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:27.731683    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7785-140008135781520"]
Jul 30 17:38:27.803689 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:27.803638    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scrb8\" (UniqueName: \"kubernetes.io/projected/7781c862-045b-4fd8-b01e-dd8dd9ebae5f-kube-api-access-scrb8\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:38:28.558910 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:28.558885    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:28.559075 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:28.558924    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e08d372dc9d75dcd460223f00ef88a981ca8431d73ba966775ee68ddfe329619"
Jul 30 17:38:28.568780 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:38:28.568748    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7785-140008135781520\" is forbidden: User \"system:node:ip-10-0-135-111.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-111.ec2.internal' and this object" podUID="7781c862-045b-4fd8-b01e-dd8dd9ebae5f" pod="opendatahub/test-curl-7785-140008135781520"
Jul 30 17:38:29.000963 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:29.000934    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7781c862-045b-4fd8-b01e-dd8dd9ebae5f" path="/var/lib/kubelet/pods/7781c862-045b-4fd8-b01e-dd8dd9ebae5f/volumes"
Jul 30 17:38:38.353440 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.353405    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"]
Jul 30 17:38:38.353980 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.353963    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7781c862-045b-4fd8-b01e-dd8dd9ebae5f" containerName="test-curl-7785-140008135781520"
Jul 30 17:38:38.354033 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.353983    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7781c862-045b-4fd8-b01e-dd8dd9ebae5f" containerName="test-curl-7785-140008135781520"
Jul 30 17:38:38.354114 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.354102    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7781c862-045b-4fd8-b01e-dd8dd9ebae5f" containerName="test-curl-7785-140008135781520"
Jul 30 17:38:38.357507 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.357486    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.360152 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.360124    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-d280ec-serving-cert\""
Jul 30 17:38:38.369456 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.369429    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"]
Jul 30 17:38:38.389348 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.389322    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p75x4\" (UniqueName: \"kubernetes.io/projected/39fec7e0-1c21-4cfd-8c58-46d682d1c922-kube-api-access-p75x4\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.389485 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.389467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.389544 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.389511    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39fec7e0-1c21-4cfd-8c58-46d682d1c922-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.490522 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.490493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.490682 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.490530    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39fec7e0-1c21-4cfd-8c58-46d682d1c922-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.490682 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.490553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p75x4\" (UniqueName: \"kubernetes.io/projected/39fec7e0-1c21-4cfd-8c58-46d682d1c922-kube-api-access-p75x4\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.490795 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:38:38.490675    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-serving-cert: secret "maas-api-e2e-ipp-a-d280ec-serving-cert" not found
Jul 30 17:38:38.490795 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:38:38.490748    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls podName:39fec7e0-1c21-4cfd-8c58-46d682d1c922 nodeName:}" failed. No retries permitted until 2026-07-30 17:38:38.990727991 +0000 UTC m=+3746.593155599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls") pod "maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn" (UID: "39fec7e0-1c21-4cfd-8c58-46d682d1c922") : secret "maas-api-e2e-ipp-a-d280ec-serving-cert" not found
Jul 30 17:38:38.491143 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.491124    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39fec7e0-1c21-4cfd-8c58-46d682d1c922-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.503413 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.503376    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p75x4\" (UniqueName: \"kubernetes.io/projected/39fec7e0-1c21-4cfd-8c58-46d682d1c922-kube-api-access-p75x4\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.994927 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:38.994896    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:38.995104 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:38:38.995048    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-serving-cert: secret "maas-api-e2e-ipp-a-d280ec-serving-cert" not found
Jul 30 17:38:38.995170 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:38:38.995138    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls podName:39fec7e0-1c21-4cfd-8c58-46d682d1c922 nodeName:}" failed. No retries permitted until 2026-07-30 17:38:39.995118819 +0000 UTC m=+3747.597546440 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls") pod "maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn" (UID: "39fec7e0-1c21-4cfd-8c58-46d682d1c922") : secret "maas-api-e2e-ipp-a-d280ec-serving-cert" not found
Jul 30 17:38:40.005232 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:40.005197    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:40.007735 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:40.007706    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:40.167463 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:40.167433    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:40.301852 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:40.301826    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"]
Jul 30 17:38:40.304073 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:38:40.304041    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39fec7e0_1c21_4cfd_8c58_46d682d1c922.slice/crio-b04aeb52d6e9db6df0f7ad92675e7c04468e060586e6671eb225c25a765d9a08 WatchSource:0}: Error finding container b04aeb52d6e9db6df0f7ad92675e7c04468e060586e6671eb225c25a765d9a08: Status 404 returned error can't find the container with id b04aeb52d6e9db6df0f7ad92675e7c04468e060586e6671eb225c25a765d9a08
Jul 30 17:38:40.613243 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:40.613169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn" event={"ID":"39fec7e0-1c21-4cfd-8c58-46d682d1c922","Type":"ContainerStarted","Data":"b04aeb52d6e9db6df0f7ad92675e7c04468e060586e6671eb225c25a765d9a08"}
Jul 30 17:38:41.619818 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:41.619785    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn" event={"ID":"39fec7e0-1c21-4cfd-8c58-46d682d1c922","Type":"ContainerStarted","Data":"9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c"}
Jul 30 17:38:41.620321 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:41.619888    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:38:41.637883 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:41.637829    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn" podStartSLOduration=2.841158126 podStartE2EDuration="3.637813589s" totalImagesPullingTime="796.655463ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:41.63639183 +0000 UTC m=+3749.238819508" watchObservedRunningTime="2026-07-30 17:38:41.637813589 +0000 UTC m=+3749.240241214"
Jul 30 17:38:47.629588 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:38:47.629558    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:39:07.506113 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.506027    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"]
Jul 30 17:39:07.518237 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.518207    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:07.519109 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.519079    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"]
Jul 30 17:39:07.522080 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.522056    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-6e5025-serving-cert\""
Jul 30 17:39:07.628193 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.628169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28117054-67cf-4592-bb1d-56194a5282eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:07.628308 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.628231    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh8h5\" (UniqueName: \"kubernetes.io/projected/28117054-67cf-4592-bb1d-56194a5282eb-kube-api-access-xh8h5\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:07.628308 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.628287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28117054-67cf-4592-bb1d-56194a5282eb-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:07.728616 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.728585    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28117054-67cf-4592-bb1d-56194a5282eb-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:07.728740 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.728658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28117054-67cf-4592-bb1d-56194a5282eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:07.728740 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.728717    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh8h5\" (UniqueName: \"kubernetes.io/projected/28117054-67cf-4592-bb1d-56194a5282eb-kube-api-access-xh8h5\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:07.728829 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:39:07.728792    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-serving-cert: secret "maas-api-e2e-ipp-b-6e5025-serving-cert" not found
Jul 30 17:39:07.728861 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:39:07.728848    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28117054-67cf-4592-bb1d-56194a5282eb-maas-api-tls podName:28117054-67cf-4592-bb1d-56194a5282eb nodeName:}" failed. No retries permitted until 2026-07-30 17:39:08.228832984 +0000 UTC m=+3775.831260589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28117054-67cf-4592-bb1d-56194a5282eb-maas-api-tls") pod "maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh" (UID: "28117054-67cf-4592-bb1d-56194a5282eb") : secret "maas-api-e2e-ipp-b-6e5025-serving-cert" not found
Jul 30 17:39:07.729163 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.729147    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28117054-67cf-4592-bb1d-56194a5282eb-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:07.738114 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:07.738088    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh8h5\" (UniqueName: \"kubernetes.io/projected/28117054-67cf-4592-bb1d-56194a5282eb-kube-api-access-xh8h5\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:08.232407 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.232372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28117054-67cf-4592-bb1d-56194a5282eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:08.234570 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.234541    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28117054-67cf-4592-bb1d-56194a5282eb-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:08.240038 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.240013    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"]
Jul 30 17:39:08.244127 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.244111    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:08.246806 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.246787    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-6e5025\""
Jul 30 17:39:08.246900 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.246832    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-6e5025-dockercfg-rdpcd\""
Jul 30 17:39:08.257789 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.257761    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"]
Jul 30 17:39:08.333656 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.333627    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pvf4\" (UniqueName: \"kubernetes.io/projected/fadea699-bd92-4f77-8e20-e2530108f822-kube-api-access-9pvf4\") pod \"payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v\" (UID: \"fadea699-bd92-4f77-8e20-e2530108f822\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:08.333768 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.333738    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fadea699-bd92-4f77-8e20-e2530108f822-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v\" (UID: \"fadea699-bd92-4f77-8e20-e2530108f822\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:08.431563 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.431537    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:08.434480 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.434142    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fadea699-bd92-4f77-8e20-e2530108f822-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v\" (UID: \"fadea699-bd92-4f77-8e20-e2530108f822\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:08.434480 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.434174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pvf4\" (UniqueName: \"kubernetes.io/projected/fadea699-bd92-4f77-8e20-e2530108f822-kube-api-access-9pvf4\") pod \"payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v\" (UID: \"fadea699-bd92-4f77-8e20-e2530108f822\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:08.434838 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.434821    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fadea699-bd92-4f77-8e20-e2530108f822-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v\" (UID: \"fadea699-bd92-4f77-8e20-e2530108f822\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:08.449270 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.449245    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pvf4\" (UniqueName: \"kubernetes.io/projected/fadea699-bd92-4f77-8e20-e2530108f822-kube-api-access-9pvf4\") pod \"payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v\" (UID: \"fadea699-bd92-4f77-8e20-e2530108f822\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:08.554733 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.554704    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:08.565265 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.565243    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"]
Jul 30 17:39:08.568476 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:39:08.568445    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28117054_67cf_4592_bb1d_56194a5282eb.slice/crio-d8fa6b713aea18aca55cb51c85fec25dc80f6da4e3cd43b059acf2fdd1ebf996 WatchSource:0}: Error finding container d8fa6b713aea18aca55cb51c85fec25dc80f6da4e3cd43b059acf2fdd1ebf996: Status 404 returned error can't find the container with id d8fa6b713aea18aca55cb51c85fec25dc80f6da4e3cd43b059acf2fdd1ebf996
Jul 30 17:39:08.692648 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.692585    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"]
Jul 30 17:39:08.693208 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:39:08.693180    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfadea699_bd92_4f77_8e20_e2530108f822.slice/crio-80125736af222f1e7cd7b4e3ec1c9b9e20b4346a5b8c7732ff4c1ea31a814b8b WatchSource:0}: Error finding container 80125736af222f1e7cd7b4e3ec1c9b9e20b4346a5b8c7732ff4c1ea31a814b8b: Status 404 returned error can't find the container with id 80125736af222f1e7cd7b4e3ec1c9b9e20b4346a5b8c7732ff4c1ea31a814b8b
Jul 30 17:39:08.742301 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.742239    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh" event={"ID":"28117054-67cf-4592-bb1d-56194a5282eb","Type":"ContainerStarted","Data":"d8fa6b713aea18aca55cb51c85fec25dc80f6da4e3cd43b059acf2fdd1ebf996"}
Jul 30 17:39:08.743797 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:08.743769    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v" event={"ID":"fadea699-bd92-4f77-8e20-e2530108f822","Type":"ContainerStarted","Data":"80125736af222f1e7cd7b4e3ec1c9b9e20b4346a5b8c7732ff4c1ea31a814b8b"}
Jul 30 17:39:09.750129 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:09.750088    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v" event={"ID":"fadea699-bd92-4f77-8e20-e2530108f822","Type":"ContainerStarted","Data":"d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b"}
Jul 30 17:39:09.750579 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:09.750148    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:09.751553 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:09.751530    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh" event={"ID":"28117054-67cf-4592-bb1d-56194a5282eb","Type":"ContainerStarted","Data":"720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29"}
Jul 30 17:39:09.751685 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:09.751665    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:09.769178 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:09.769132    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v" podStartSLOduration=1.7691201460000001 podStartE2EDuration="1.769120146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:39:09.768556889 +0000 UTC m=+3777.370984516" watchObservedRunningTime="2026-07-30 17:39:09.769120146 +0000 UTC m=+3777.371547766"
Jul 30 17:39:09.786639 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:09.786586    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh" podStartSLOduration=2.463208993 podStartE2EDuration="2.786573908s" totalImagesPullingTime="323.364915ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:39:09.78525643 +0000 UTC m=+3777.387684056" watchObservedRunningTime="2026-07-30 17:39:09.786573908 +0000 UTC m=+3777.389001534"
Jul 30 17:39:15.761918 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:15.761891    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:39:20.758343 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:20.758315    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:39:38.350180 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:39:38.350150    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wgxkq_1d438334-8a53-4796-8080-8ce8dabe3afc/payload-processing/0.log"
Jul 30 17:40:28.466232 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:28.466203    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wgxkq_1d438334-8a53-4796-8080-8ce8dabe3afc/payload-processing/0.log"
Jul 30 17:40:33.194083 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.194043    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"]
Jul 30 17:40:33.197658 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.197636    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.200190 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.200171    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-ea2bbf-serving-cert\""
Jul 30 17:40:33.212300 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.212272    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"]
Jul 30 17:40:33.301200 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.301172    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.301301 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.301211    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42mgc\" (UniqueName: \"kubernetes.io/projected/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-kube-api-access-42mgc\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.301343 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.301318    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.401901 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.401876    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42mgc\" (UniqueName: \"kubernetes.io/projected/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-kube-api-access-42mgc\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.402007 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.401937    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.402007 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.401984    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.402127 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:40:33.402062    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-serving-cert: secret "maas-api-e2e-ipp-cleanup-ea2bbf-serving-cert" not found
Jul 30 17:40:33.402127 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:40:33.402118    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-maas-api-tls podName:f0079f2f-feb1-44b0-8fba-ac6cea2e541c nodeName:}" failed. No retries permitted until 2026-07-30 17:40:33.902102942 +0000 UTC m=+3861.504530547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd" (UID: "f0079f2f-feb1-44b0-8fba-ac6cea2e541c") : secret "maas-api-e2e-ipp-cleanup-ea2bbf-serving-cert" not found
Jul 30 17:40:33.402568 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.402547    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.413586 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.413559    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42mgc\" (UniqueName: \"kubernetes.io/projected/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-kube-api-access-42mgc\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.906821 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.906786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:33.909166 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:33.909138    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:34.107518 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:34.107486    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:34.255978 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:34.255953    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"]
Jul 30 17:40:34.258160 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:40:34.258132    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0079f2f_feb1_44b0_8fba_ac6cea2e541c.slice/crio-8e18f2761c203f06d0fa48528ca43f92878b51946a307a4d6910281600161755 WatchSource:0}: Error finding container 8e18f2761c203f06d0fa48528ca43f92878b51946a307a4d6910281600161755: Status 404 returned error can't find the container with id 8e18f2761c203f06d0fa48528ca43f92878b51946a307a4d6910281600161755
Jul 30 17:40:35.148933 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:35.148903    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd" event={"ID":"f0079f2f-feb1-44b0-8fba-ac6cea2e541c","Type":"ContainerStarted","Data":"90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188"}
Jul 30 17:40:35.149118 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:35.148959    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd" event={"ID":"f0079f2f-feb1-44b0-8fba-ac6cea2e541c","Type":"ContainerStarted","Data":"8e18f2761c203f06d0fa48528ca43f92878b51946a307a4d6910281600161755"}
Jul 30 17:40:35.149118 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:35.148989    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:35.166832 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:35.166790    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd" podStartSLOduration=1.59866462 podStartE2EDuration="2.166774521s" totalImagesPullingTime="568.109901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:35.165439442 +0000 UTC m=+3862.767867071" watchObservedRunningTime="2026-07-30 17:40:35.166774521 +0000 UTC m=+3862.769202147"
Jul 30 17:40:41.160332 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:41.160305    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:40:57.628084 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:57.628049    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"]
Jul 30 17:40:57.632058 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:57.632039    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"
Jul 30 17:40:57.634789 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:57.634771    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:40:57.638432 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:57.638411    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"]
Jul 30 17:40:57.701489 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:57.701455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057b3299-4b07-4b98-8bce-398f9cd7b69a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx\" (UID: \"057b3299-4b07-4b98-8bce-398f9cd7b69a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"
Jul 30 17:40:57.802234 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:57.802203    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057b3299-4b07-4b98-8bce-398f9cd7b69a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx\" (UID: \"057b3299-4b07-4b98-8bce-398f9cd7b69a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"
Jul 30 17:40:57.802808 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:57.802793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057b3299-4b07-4b98-8bce-398f9cd7b69a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx\" (UID: \"057b3299-4b07-4b98-8bce-398f9cd7b69a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"
Jul 30 17:40:57.945066 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:57.945035    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"
Jul 30 17:40:58.073728 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:58.073693    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"]
Jul 30 17:40:58.076060 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:40:58.076032    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057b3299_4b07_4b98_8bce_398f9cd7b69a.slice/crio-5a9d92d40d351b66ff6e79deee517e6992ee8b3c84476f29bcd9e3e785cb46b6 WatchSource:0}: Error finding container 5a9d92d40d351b66ff6e79deee517e6992ee8b3c84476f29bcd9e3e785cb46b6: Status 404 returned error can't find the container with id 5a9d92d40d351b66ff6e79deee517e6992ee8b3c84476f29bcd9e3e785cb46b6
Jul 30 17:40:58.253285 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:58.253222    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx" event={"ID":"057b3299-4b07-4b98-8bce-398f9cd7b69a","Type":"ContainerStarted","Data":"5a9d92d40d351b66ff6e79deee517e6992ee8b3c84476f29bcd9e3e785cb46b6"}
Jul 30 17:40:59.258877 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:59.258844    2569 generic.go:356] "Generic (PLEG): container finished" podID="057b3299-4b07-4b98-8bce-398f9cd7b69a" containerID="bbdb54f7ea88d20d22aaca8b17d7a0c302fa780d1347d50e8ffacc4255d3b2ba" exitCode=0
Jul 30 17:40:59.259274 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:40:59.258917    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx" event={"ID":"057b3299-4b07-4b98-8bce-398f9cd7b69a","Type":"ContainerDied","Data":"bbdb54f7ea88d20d22aaca8b17d7a0c302fa780d1347d50e8ffacc4255d3b2ba"}
Jul 30 17:41:00.385176 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:00.385152    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"
Jul 30 17:41:00.422822 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:00.422794    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057b3299-4b07-4b98-8bce-398f9cd7b69a-maas-api-service-ca\") pod \"057b3299-4b07-4b98-8bce-398f9cd7b69a\" (UID: \"057b3299-4b07-4b98-8bce-398f9cd7b69a\") "
Jul 30 17:41:00.423226 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:00.423202    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/057b3299-4b07-4b98-8bce-398f9cd7b69a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "057b3299-4b07-4b98-8bce-398f9cd7b69a" (UID: "057b3299-4b07-4b98-8bce-398f9cd7b69a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:41:00.524458 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:00.524401    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/057b3299-4b07-4b98-8bce-398f9cd7b69a-maas-api-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:41:01.269678 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:01.269600    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"
Jul 30 17:41:01.269678 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:01.269641    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx" event={"ID":"057b3299-4b07-4b98-8bce-398f9cd7b69a","Type":"ContainerDied","Data":"5a9d92d40d351b66ff6e79deee517e6992ee8b3c84476f29bcd9e3e785cb46b6"}
Jul 30 17:41:01.269836 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:01.269684    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a9d92d40d351b66ff6e79deee517e6992ee8b3c84476f29bcd9e3e785cb46b6"
Jul 30 17:41:03.583900 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.583862    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"]
Jul 30 17:41:03.584321 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.584096    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd" podUID="f0079f2f-feb1-44b0-8fba-ac6cea2e541c" containerName="maas-api" containerID="cri-o://90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188" gracePeriod=30
Jul 30 17:41:03.785696 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.785666    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"]
Jul 30 17:41:03.793377 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.793347    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ea2bbf-c2wwx"]
Jul 30 17:41:03.834440 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.834380    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:41:03.954059 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.954031    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-maas-api-tls\") pod \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") "
Jul 30 17:41:03.954059 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.954064    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42mgc\" (UniqueName: \"kubernetes.io/projected/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-kube-api-access-42mgc\") pod \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") "
Jul 30 17:41:03.954293 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.954085    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-openshift-service-ca\") pod \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\" (UID: \"f0079f2f-feb1-44b0-8fba-ac6cea2e541c\") "
Jul 30 17:41:03.954529 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.954496    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f0079f2f-feb1-44b0-8fba-ac6cea2e541c" (UID: "f0079f2f-feb1-44b0-8fba-ac6cea2e541c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:41:03.956128 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.956097    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f0079f2f-feb1-44b0-8fba-ac6cea2e541c" (UID: "f0079f2f-feb1-44b0-8fba-ac6cea2e541c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:41:03.956235 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:03.956212    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-kube-api-access-42mgc" (OuterVolumeSpecName: "kube-api-access-42mgc") pod "f0079f2f-feb1-44b0-8fba-ac6cea2e541c" (UID: "f0079f2f-feb1-44b0-8fba-ac6cea2e541c"). InnerVolumeSpecName "kube-api-access-42mgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:41:04.055531 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.055508    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:41:04.055531 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.055530    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42mgc\" (UniqueName: \"kubernetes.io/projected/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-kube-api-access-42mgc\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:41:04.055722 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.055540    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0079f2f-feb1-44b0-8fba-ac6cea2e541c-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:41:04.285327 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.285300    2569 generic.go:356] "Generic (PLEG): container finished" podID="f0079f2f-feb1-44b0-8fba-ac6cea2e541c" containerID="90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188" exitCode=0
Jul 30 17:41:04.285550 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.285339    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"
Jul 30 17:41:04.285550 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.285380    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd" event={"ID":"f0079f2f-feb1-44b0-8fba-ac6cea2e541c","Type":"ContainerDied","Data":"90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188"}
Jul 30 17:41:04.285550 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.285422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd" event={"ID":"f0079f2f-feb1-44b0-8fba-ac6cea2e541c","Type":"ContainerDied","Data":"8e18f2761c203f06d0fa48528ca43f92878b51946a307a4d6910281600161755"}
Jul 30 17:41:04.285550 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.285446    2569 scope.go:117] "RemoveContainer" containerID="90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188"
Jul 30 17:41:04.294696 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.294679    2569 scope.go:117] "RemoveContainer" containerID="90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188"
Jul 30 17:41:04.294929 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:41:04.294915    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188\": container with ID starting with 90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188 not found: ID does not exist" containerID="90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188"
Jul 30 17:41:04.294979 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.294936    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188"} err="failed to get container status \"90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188\": rpc error: code = NotFound desc = could not find container \"90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188\": container with ID starting with 90e463e44f7ac288df5d92497029f71f5a2d97dcb4fbbac991ffafa9e715f188 not found: ID does not exist"
Jul 30 17:41:04.308122 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.308100    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"]
Jul 30 17:41:04.312502 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:04.312481    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ea2bbf-58467cf5d4-dssmd"]
Jul 30 17:41:05.001729 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:05.001699    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="057b3299-4b07-4b98-8bce-398f9cd7b69a" path="/var/lib/kubelet/pods/057b3299-4b07-4b98-8bce-398f9cd7b69a/volumes"
Jul 30 17:41:05.002119 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:05.002109    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0079f2f-feb1-44b0-8fba-ac6cea2e541c" path="/var/lib/kubelet/pods/f0079f2f-feb1-44b0-8fba-ac6cea2e541c/volumes"
Jul 30 17:41:14.098808 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.098777    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"]
Jul 30 17:41:14.099183 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.099170    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="057b3299-4b07-4b98-8bce-398f9cd7b69a" containerName="revoke-keys"
Jul 30 17:41:14.099183 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.099184    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="057b3299-4b07-4b98-8bce-398f9cd7b69a" containerName="revoke-keys"
Jul 30 17:41:14.099262 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.099201    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0079f2f-feb1-44b0-8fba-ac6cea2e541c" containerName="maas-api"
Jul 30 17:41:14.099262 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.099206    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0079f2f-feb1-44b0-8fba-ac6cea2e541c" containerName="maas-api"
Jul 30 17:41:14.099324 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.099266    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="057b3299-4b07-4b98-8bce-398f9cd7b69a" containerName="revoke-keys"
Jul 30 17:41:14.099324 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.099274    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0079f2f-feb1-44b0-8fba-ac6cea2e541c" containerName="maas-api"
Jul 30 17:41:14.103989 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.103972    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"
Jul 30 17:41:14.106796 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.106773    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:41:14.109455 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.109416    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"]
Jul 30 17:41:14.238200 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.238161    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06db7fca-36e8-4f83-b6a1-19316eaade51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj\" (UID: \"06db7fca-36e8-4f83-b6a1-19316eaade51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"
Jul 30 17:41:14.339577 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.339550    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06db7fca-36e8-4f83-b6a1-19316eaade51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj\" (UID: \"06db7fca-36e8-4f83-b6a1-19316eaade51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"
Jul 30 17:41:14.340147 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.340128    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06db7fca-36e8-4f83-b6a1-19316eaade51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj\" (UID: \"06db7fca-36e8-4f83-b6a1-19316eaade51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"
Jul 30 17:41:14.417129 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.417110    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"
Jul 30 17:41:14.546011 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:14.545978    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"]
Jul 30 17:41:14.549596 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:41:14.549566    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06db7fca_36e8_4f83_b6a1_19316eaade51.slice/crio-3118d52bb1071aef7b4a9603d41ea378150c1d5c1a8c1546fad7ace5cf510084 WatchSource:0}: Error finding container 3118d52bb1071aef7b4a9603d41ea378150c1d5c1a8c1546fad7ace5cf510084: Status 404 returned error can't find the container with id 3118d52bb1071aef7b4a9603d41ea378150c1d5c1a8c1546fad7ace5cf510084
Jul 30 17:41:15.342690 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:15.342638    2569 generic.go:356] "Generic (PLEG): container finished" podID="06db7fca-36e8-4f83-b6a1-19316eaade51" containerID="34902f1bb07c0f6fa471f9c4b542bd7ac7fa5fcccb8e36ddb4dacffe6394c09a" exitCode=0
Jul 30 17:41:15.343048 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:15.342724    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj" event={"ID":"06db7fca-36e8-4f83-b6a1-19316eaade51","Type":"ContainerDied","Data":"34902f1bb07c0f6fa471f9c4b542bd7ac7fa5fcccb8e36ddb4dacffe6394c09a"}
Jul 30 17:41:15.343048 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:15.342784    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj" event={"ID":"06db7fca-36e8-4f83-b6a1-19316eaade51","Type":"ContainerStarted","Data":"3118d52bb1071aef7b4a9603d41ea378150c1d5c1a8c1546fad7ace5cf510084"}
Jul 30 17:41:16.478090 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:16.478064    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"
Jul 30 17:41:16.558509 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:16.558483    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06db7fca-36e8-4f83-b6a1-19316eaade51-maas-api-service-ca\") pod \"06db7fca-36e8-4f83-b6a1-19316eaade51\" (UID: \"06db7fca-36e8-4f83-b6a1-19316eaade51\") "
Jul 30 17:41:16.558868 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:16.558848    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06db7fca-36e8-4f83-b6a1-19316eaade51-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06db7fca-36e8-4f83-b6a1-19316eaade51" (UID: "06db7fca-36e8-4f83-b6a1-19316eaade51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:41:16.660047 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:16.660011    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06db7fca-36e8-4f83-b6a1-19316eaade51-maas-api-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:41:17.353363 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:17.353273    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"
Jul 30 17:41:17.353363 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:17.353295    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj" event={"ID":"06db7fca-36e8-4f83-b6a1-19316eaade51","Type":"ContainerDied","Data":"3118d52bb1071aef7b4a9603d41ea378150c1d5c1a8c1546fad7ace5cf510084"}
Jul 30 17:41:17.353363 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:17.353343    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3118d52bb1071aef7b4a9603d41ea378150c1d5c1a8c1546fad7ace5cf510084"
Jul 30 17:41:29.656798 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:29.656763    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"]
Jul 30 17:41:29.657297 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:29.657033    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn" podUID="39fec7e0-1c21-4cfd-8c58-46d682d1c922" containerName="maas-api" containerID="cri-o://9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c" gracePeriod=30
Jul 30 17:41:29.850546 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:29.850511    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"]
Jul 30 17:41:29.854638 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:29.854598    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d280ec-l77nj"]
Jul 30 17:41:29.895471 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:29.895450    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:41:30.071358 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.071294    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39fec7e0-1c21-4cfd-8c58-46d682d1c922-openshift-service-ca\") pod \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") "
Jul 30 17:41:30.071475 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.071373    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p75x4\" (UniqueName: \"kubernetes.io/projected/39fec7e0-1c21-4cfd-8c58-46d682d1c922-kube-api-access-p75x4\") pod \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") "
Jul 30 17:41:30.071475 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.071440    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls\") pod \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\" (UID: \"39fec7e0-1c21-4cfd-8c58-46d682d1c922\") "
Jul 30 17:41:30.071701 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.071674    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39fec7e0-1c21-4cfd-8c58-46d682d1c922-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39fec7e0-1c21-4cfd-8c58-46d682d1c922" (UID: "39fec7e0-1c21-4cfd-8c58-46d682d1c922"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:41:30.073338 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.073314    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39fec7e0-1c21-4cfd-8c58-46d682d1c922" (UID: "39fec7e0-1c21-4cfd-8c58-46d682d1c922"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:41:30.073338 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.073332    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39fec7e0-1c21-4cfd-8c58-46d682d1c922-kube-api-access-p75x4" (OuterVolumeSpecName: "kube-api-access-p75x4") pod "39fec7e0-1c21-4cfd-8c58-46d682d1c922" (UID: "39fec7e0-1c21-4cfd-8c58-46d682d1c922"). InnerVolumeSpecName "kube-api-access-p75x4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:41:30.172042 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.172014    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p75x4\" (UniqueName: \"kubernetes.io/projected/39fec7e0-1c21-4cfd-8c58-46d682d1c922-kube-api-access-p75x4\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:41:30.172042 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.172038    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39fec7e0-1c21-4cfd-8c58-46d682d1c922-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:41:30.172192 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.172049    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39fec7e0-1c21-4cfd-8c58-46d682d1c922-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:41:30.416826 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.416791    2569 generic.go:356] "Generic (PLEG): container finished" podID="39fec7e0-1c21-4cfd-8c58-46d682d1c922" containerID="9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c" exitCode=0
Jul 30 17:41:30.416980 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.416840    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"
Jul 30 17:41:30.416980 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.416872    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn" event={"ID":"39fec7e0-1c21-4cfd-8c58-46d682d1c922","Type":"ContainerDied","Data":"9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c"}
Jul 30 17:41:30.416980 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.416923    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn" event={"ID":"39fec7e0-1c21-4cfd-8c58-46d682d1c922","Type":"ContainerDied","Data":"b04aeb52d6e9db6df0f7ad92675e7c04468e060586e6671eb225c25a765d9a08"}
Jul 30 17:41:30.416980 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.416943    2569 scope.go:117] "RemoveContainer" containerID="9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c"
Jul 30 17:41:30.425799 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.425783    2569 scope.go:117] "RemoveContainer" containerID="9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c"
Jul 30 17:41:30.426045 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:41:30.426031    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c\": container with ID starting with 9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c not found: ID does not exist" containerID="9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c"
Jul 30 17:41:30.426086 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.426052    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c"} err="failed to get container status \"9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c\": rpc error: code = NotFound desc = could not find container \"9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c\": container with ID starting with 9a1cfda7f5f40f3ca217dd6ec1ef3223f61aa2bc9ed78c580eda5fc7650b250c not found: ID does not exist"
Jul 30 17:41:30.442093 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.442068    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"]
Jul 30 17:41:30.448576 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:30.448557    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d280ec-665d95477c-s6qxn"]
Jul 30 17:41:31.000919 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:31.000886    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06db7fca-36e8-4f83-b6a1-19316eaade51" path="/var/lib/kubelet/pods/06db7fca-36e8-4f83-b6a1-19316eaade51/volumes"
Jul 30 17:41:31.001323 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:41:31.001277    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39fec7e0-1c21-4cfd-8c58-46d682d1c922" path="/var/lib/kubelet/pods/39fec7e0-1c21-4cfd-8c58-46d682d1c922/volumes"
Jul 30 17:42:43.895453 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:43.895413    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"]
Jul 30 17:42:43.896037 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:43.895669    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh" podUID="28117054-67cf-4592-bb1d-56194a5282eb" containerName="maas-api" containerID="cri-o://720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29" gracePeriod=30
Jul 30 17:42:43.965852 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:43.965824    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"]
Jul 30 17:42:43.966026 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:43.966002    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v" podUID="fadea699-bd92-4f77-8e20-e2530108f822" containerName="payload-pre-processing" containerID="cri-o://d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b" gracePeriod=30
Jul 30 17:42:44.225081 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.225056    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:42:44.249510 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.249489    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:42:44.366843 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.366813    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fadea699-bd92-4f77-8e20-e2530108f822-plugins-config-volume\") pod \"fadea699-bd92-4f77-8e20-e2530108f822\" (UID: \"fadea699-bd92-4f77-8e20-e2530108f822\") "
Jul 30 17:42:44.366973 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.366859    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh8h5\" (UniqueName: \"kubernetes.io/projected/28117054-67cf-4592-bb1d-56194a5282eb-kube-api-access-xh8h5\") pod \"28117054-67cf-4592-bb1d-56194a5282eb\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") "
Jul 30 17:42:44.366973 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.366882    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pvf4\" (UniqueName: \"kubernetes.io/projected/fadea699-bd92-4f77-8e20-e2530108f822-kube-api-access-9pvf4\") pod \"fadea699-bd92-4f77-8e20-e2530108f822\" (UID: \"fadea699-bd92-4f77-8e20-e2530108f822\") "
Jul 30 17:42:44.366973 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.366919    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28117054-67cf-4592-bb1d-56194a5282eb-maas-api-tls\") pod \"28117054-67cf-4592-bb1d-56194a5282eb\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") "
Jul 30 17:42:44.366973 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.366961    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28117054-67cf-4592-bb1d-56194a5282eb-openshift-service-ca\") pod \"28117054-67cf-4592-bb1d-56194a5282eb\" (UID: \"28117054-67cf-4592-bb1d-56194a5282eb\") "
Jul 30 17:42:44.367174 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.367146    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fadea699-bd92-4f77-8e20-e2530108f822-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fadea699-bd92-4f77-8e20-e2530108f822" (UID: "fadea699-bd92-4f77-8e20-e2530108f822"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:42:44.367384 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.367362    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fadea699-bd92-4f77-8e20-e2530108f822-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:42:44.367506 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.367402    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28117054-67cf-4592-bb1d-56194a5282eb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28117054-67cf-4592-bb1d-56194a5282eb" (UID: "28117054-67cf-4592-bb1d-56194a5282eb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:42:44.369034 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.369007    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28117054-67cf-4592-bb1d-56194a5282eb-kube-api-access-xh8h5" (OuterVolumeSpecName: "kube-api-access-xh8h5") pod "28117054-67cf-4592-bb1d-56194a5282eb" (UID: "28117054-67cf-4592-bb1d-56194a5282eb"). InnerVolumeSpecName "kube-api-access-xh8h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:42:44.369034 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.369026    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28117054-67cf-4592-bb1d-56194a5282eb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28117054-67cf-4592-bb1d-56194a5282eb" (UID: "28117054-67cf-4592-bb1d-56194a5282eb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:42:44.369161 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.369037    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fadea699-bd92-4f77-8e20-e2530108f822-kube-api-access-9pvf4" (OuterVolumeSpecName: "kube-api-access-9pvf4") pod "fadea699-bd92-4f77-8e20-e2530108f822" (UID: "fadea699-bd92-4f77-8e20-e2530108f822"). InnerVolumeSpecName "kube-api-access-9pvf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:42:44.468166 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.468110    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh8h5\" (UniqueName: \"kubernetes.io/projected/28117054-67cf-4592-bb1d-56194a5282eb-kube-api-access-xh8h5\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:42:44.468166 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.468134    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pvf4\" (UniqueName: \"kubernetes.io/projected/fadea699-bd92-4f77-8e20-e2530108f822-kube-api-access-9pvf4\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:42:44.468166 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.468144    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28117054-67cf-4592-bb1d-56194a5282eb-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:42:44.468166 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.468152    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28117054-67cf-4592-bb1d-56194a5282eb-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:42:44.775560 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.775498    2569 generic.go:356] "Generic (PLEG): container finished" podID="28117054-67cf-4592-bb1d-56194a5282eb" containerID="720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29" exitCode=0
Jul 30 17:42:44.775560 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.775540    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"
Jul 30 17:42:44.775813 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.775585    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh" event={"ID":"28117054-67cf-4592-bb1d-56194a5282eb","Type":"ContainerDied","Data":"720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29"}
Jul 30 17:42:44.775813 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.775667    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh" event={"ID":"28117054-67cf-4592-bb1d-56194a5282eb","Type":"ContainerDied","Data":"d8fa6b713aea18aca55cb51c85fec25dc80f6da4e3cd43b059acf2fdd1ebf996"}
Jul 30 17:42:44.775813 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.775700    2569 scope.go:117] "RemoveContainer" containerID="720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29"
Jul 30 17:42:44.777001 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.776972    2569 generic.go:356] "Generic (PLEG): container finished" podID="fadea699-bd92-4f77-8e20-e2530108f822" containerID="d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b" exitCode=0
Jul 30 17:42:44.777137 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.776997    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v" event={"ID":"fadea699-bd92-4f77-8e20-e2530108f822","Type":"ContainerDied","Data":"d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b"}
Jul 30 17:42:44.777137 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.777021    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"
Jul 30 17:42:44.777137 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.777041    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v" event={"ID":"fadea699-bd92-4f77-8e20-e2530108f822","Type":"ContainerDied","Data":"80125736af222f1e7cd7b4e3ec1c9b9e20b4346a5b8c7732ff4c1ea31a814b8b"}
Jul 30 17:42:44.785288 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.785146    2569 scope.go:117] "RemoveContainer" containerID="720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29"
Jul 30 17:42:44.785519 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:42:44.785495    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29\": container with ID starting with 720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29 not found: ID does not exist" containerID="720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29"
Jul 30 17:42:44.785592 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.785528    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29"} err="failed to get container status \"720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29\": rpc error: code = NotFound desc = could not find container \"720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29\": container with ID starting with 720a1908fc5d57a72bb72f9ea94db53d4dfcb5b337fe05b8b4ba9a7a5fb34b29 not found: ID does not exist"
Jul 30 17:42:44.785592 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.785550    2569 scope.go:117] "RemoveContainer" containerID="d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b"
Jul 30 17:42:44.793536 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.793518    2569 scope.go:117] "RemoveContainer" containerID="d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b"
Jul 30 17:42:44.793870 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:42:44.793852    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b\": container with ID starting with d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b not found: ID does not exist" containerID="d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b"
Jul 30 17:42:44.793925 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.793876    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b"} err="failed to get container status \"d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b\": rpc error: code = NotFound desc = could not find container \"d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b\": container with ID starting with d8cdd3f5482e17c889c79b234a0a5313234450a3b68e301592974be4f9f7ec4b not found: ID does not exist"
Jul 30 17:42:44.807129 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.807108    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"]
Jul 30 17:42:44.813753 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.813732    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6e5025-959467b9b-vtt7v"]
Jul 30 17:42:44.826148 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.826126    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"]
Jul 30 17:42:44.831339 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:44.831317    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6e5025-67998d767c-gcjgh"]
Jul 30 17:42:45.000613 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:45.000588    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28117054-67cf-4592-bb1d-56194a5282eb" path="/var/lib/kubelet/pods/28117054-67cf-4592-bb1d-56194a5282eb/volumes"
Jul 30 17:42:45.000963 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:42:45.000949    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fadea699-bd92-4f77-8e20-e2530108f822" path="/var/lib/kubelet/pods/fadea699-bd92-4f77-8e20-e2530108f822/volumes"
Jul 30 17:43:01.303481 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.303425    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"]
Jul 30 17:43:01.304250 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.304220    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65" podUID="3a211841-3798-4dd0-8ce9-bac2ab04c310" containerName="maas-api" containerID="cri-o://79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84" gracePeriod=30
Jul 30 17:43:01.354413 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.354378    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65" podUID="3a211841-3798-4dd0-8ce9-bac2ab04c310" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.65:8443/health\": dial tcp 10.134.0.65:8443: connect: connection refused"
Jul 30 17:43:01.358656 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.358626    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"]
Jul 30 17:43:01.358914 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.358888    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj" podUID="9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea" containerName="payload-processing" containerID="cri-o://50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c" gracePeriod=30
Jul 30 17:43:01.601878 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.601855    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:43:01.630123 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.630102    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:43:01.700508 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.700480    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-plugins-config-volume\") pod \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\" (UID: \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\") "
Jul 30 17:43:01.700655 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.700548    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a211841-3798-4dd0-8ce9-bac2ab04c310-maas-api-tls\") pod \"3a211841-3798-4dd0-8ce9-bac2ab04c310\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") "
Jul 30 17:43:01.700655 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.700602    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a211841-3798-4dd0-8ce9-bac2ab04c310-openshift-service-ca\") pod \"3a211841-3798-4dd0-8ce9-bac2ab04c310\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") "
Jul 30 17:43:01.700655 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.700653    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zb6sw\" (UniqueName: \"kubernetes.io/projected/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-kube-api-access-zb6sw\") pod \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\" (UID: \"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea\") "
Jul 30 17:43:01.700762 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.700673    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsnlc\" (UniqueName: \"kubernetes.io/projected/3a211841-3798-4dd0-8ce9-bac2ab04c310-kube-api-access-vsnlc\") pod \"3a211841-3798-4dd0-8ce9-bac2ab04c310\" (UID: \"3a211841-3798-4dd0-8ce9-bac2ab04c310\") "
Jul 30 17:43:01.700912 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.700886    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea" (UID: "9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:43:01.701091 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.701068    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a211841-3798-4dd0-8ce9-bac2ab04c310-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3a211841-3798-4dd0-8ce9-bac2ab04c310" (UID: "3a211841-3798-4dd0-8ce9-bac2ab04c310"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:43:01.702633 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.702596    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a211841-3798-4dd0-8ce9-bac2ab04c310-kube-api-access-vsnlc" (OuterVolumeSpecName: "kube-api-access-vsnlc") pod "3a211841-3798-4dd0-8ce9-bac2ab04c310" (UID: "3a211841-3798-4dd0-8ce9-bac2ab04c310"). InnerVolumeSpecName "kube-api-access-vsnlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:43:01.702961 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.702932    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a211841-3798-4dd0-8ce9-bac2ab04c310-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3a211841-3798-4dd0-8ce9-bac2ab04c310" (UID: "3a211841-3798-4dd0-8ce9-bac2ab04c310"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:43:01.703056 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.702982    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-kube-api-access-zb6sw" (OuterVolumeSpecName: "kube-api-access-zb6sw") pod "9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea" (UID: "9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea"). InnerVolumeSpecName "kube-api-access-zb6sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:43:01.801750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.801717    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a211841-3798-4dd0-8ce9-bac2ab04c310-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:43:01.801750 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.801750    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb6sw\" (UniqueName: \"kubernetes.io/projected/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-kube-api-access-zb6sw\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:43:01.801887 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.801763    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsnlc\" (UniqueName: \"kubernetes.io/projected/3a211841-3798-4dd0-8ce9-bac2ab04c310-kube-api-access-vsnlc\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:43:01.801887 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.801776    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea-plugins-config-volume\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:43:01.801887 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.801789    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a211841-3798-4dd0-8ce9-bac2ab04c310-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:43:01.860248 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.860187    2569 generic.go:356] "Generic (PLEG): container finished" podID="9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea" containerID="50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c" exitCode=0
Jul 30 17:43:01.860248 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.860230    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"
Jul 30 17:43:01.860387 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.860271    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj" event={"ID":"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea","Type":"ContainerDied","Data":"50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c"}
Jul 30 17:43:01.860387 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.860321    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj" event={"ID":"9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea","Type":"ContainerDied","Data":"0292acc073270db67f27f62242fa7c31cd2552badadc5e653ca6f8195c094c2e"}
Jul 30 17:43:01.860387 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.860344    2569 scope.go:117] "RemoveContainer" containerID="50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c"
Jul 30 17:43:01.861399 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.861379    2569 generic.go:356] "Generic (PLEG): container finished" podID="3a211841-3798-4dd0-8ce9-bac2ab04c310" containerID="79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84" exitCode=0
Jul 30 17:43:01.861498 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.861410    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65" event={"ID":"3a211841-3798-4dd0-8ce9-bac2ab04c310","Type":"ContainerDied","Data":"79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84"}
Jul 30 17:43:01.861498 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.861425    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"
Jul 30 17:43:01.861498 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.861460    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65" event={"ID":"3a211841-3798-4dd0-8ce9-bac2ab04c310","Type":"ContainerDied","Data":"c19dec5b476e5853f133354e984cd095064d4da16960e335d7f7edaab2522db5"}
Jul 30 17:43:01.868543 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.868526    2569 scope.go:117] "RemoveContainer" containerID="50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c"
Jul 30 17:43:01.868822 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:43:01.868805    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c\": container with ID starting with 50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c not found: ID does not exist" containerID="50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c"
Jul 30 17:43:01.868887 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.868829    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c"} err="failed to get container status \"50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c\": rpc error: code = NotFound desc = could not find container \"50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c\": container with ID starting with 50b0a3b817a136b5cf06838d6a022f6e9ea2940cc3b10551966a77b6a454c45c not found: ID does not exist"
Jul 30 17:43:01.868887 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.868845    2569 scope.go:117] "RemoveContainer" containerID="79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84"
Jul 30 17:43:01.876543 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.876526    2569 scope.go:117] "RemoveContainer" containerID="79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84"
Jul 30 17:43:01.877029 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:43:01.876992    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84\": container with ID starting with 79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84 not found: ID does not exist" containerID="79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84"
Jul 30 17:43:01.877136 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.877038    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84"} err="failed to get container status \"79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84\": rpc error: code = NotFound desc = could not find container \"79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84\": container with ID starting with 79946a74790671eb3eb4d58630a2a0f55773bf39d71ed0707b02d22d8caf6f84 not found: ID does not exist"
Jul 30 17:43:01.887926 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.887903    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"]
Jul 30 17:43:01.890984 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.890966    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f45123-6fd64c8878-c22jj"]
Jul 30 17:43:01.904215 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.904197    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"]
Jul 30 17:43:01.908678 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:01.908659    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-f45123-74776fb64-fjl65"]
Jul 30 17:43:03.005941 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:03.005910    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a211841-3798-4dd0-8ce9-bac2ab04c310" path="/var/lib/kubelet/pods/3a211841-3798-4dd0-8ce9-bac2ab04c310/volumes"
Jul 30 17:43:03.006297 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:03.006233    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea" path="/var/lib/kubelet/pods/9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea/volumes"
Jul 30 17:43:08.694943 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.694904    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"]
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695267    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28117054-67cf-4592-bb1d-56194a5282eb" containerName="maas-api"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695279    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="28117054-67cf-4592-bb1d-56194a5282eb" containerName="maas-api"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695290    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a211841-3798-4dd0-8ce9-bac2ab04c310" containerName="maas-api"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695295    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a211841-3798-4dd0-8ce9-bac2ab04c310" containerName="maas-api"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695310    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea" containerName="payload-processing"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695318    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea" containerName="payload-processing"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695327    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39fec7e0-1c21-4cfd-8c58-46d682d1c922" containerName="maas-api"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695333    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39fec7e0-1c21-4cfd-8c58-46d682d1c922" containerName="maas-api"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695338    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fadea699-bd92-4f77-8e20-e2530108f822" containerName="payload-pre-processing"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695344    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fadea699-bd92-4f77-8e20-e2530108f822" containerName="payload-pre-processing"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695359    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06db7fca-36e8-4f83-b6a1-19316eaade51" containerName="revoke-keys"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695364    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="06db7fca-36e8-4f83-b6a1-19316eaade51" containerName="revoke-keys"
Jul 30 17:43:08.695421 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695424    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aa95483-f68f-4bf0-97b5-2ef6cfb7c1ea" containerName="payload-processing"
Jul 30 17:43:08.695942 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695434    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="06db7fca-36e8-4f83-b6a1-19316eaade51" containerName="revoke-keys"
Jul 30 17:43:08.695942 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695440    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39fec7e0-1c21-4cfd-8c58-46d682d1c922" containerName="maas-api"
Jul 30 17:43:08.695942 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695445    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="28117054-67cf-4592-bb1d-56194a5282eb" containerName="maas-api"
Jul 30 17:43:08.695942 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695451    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fadea699-bd92-4f77-8e20-e2530108f822" containerName="payload-pre-processing"
Jul 30 17:43:08.695942 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.695458    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a211841-3798-4dd0-8ce9-bac2ab04c310" containerName="maas-api"
Jul 30 17:43:08.699305 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.699287    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"
Jul 30 17:43:08.702098 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.702078    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfhgn\""
Jul 30 17:43:08.707054 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.707036    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"]
Jul 30 17:43:08.752975 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.752955    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e998ea-a369-4a9e-bce7-fe64925cb788-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz\" (UID: \"a6e998ea-a369-4a9e-bce7-fe64925cb788\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"
Jul 30 17:43:08.853802 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.853780    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e998ea-a369-4a9e-bce7-fe64925cb788-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz\" (UID: \"a6e998ea-a369-4a9e-bce7-fe64925cb788\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"
Jul 30 17:43:08.854317 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:08.854301    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e998ea-a369-4a9e-bce7-fe64925cb788-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz\" (UID: \"a6e998ea-a369-4a9e-bce7-fe64925cb788\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"
Jul 30 17:43:09.010138 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:09.010079    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"
Jul 30 17:43:09.140951 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:09.140924    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"]
Jul 30 17:43:09.143762 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:43:09.143729    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e998ea_a369_4a9e_bce7_fe64925cb788.slice/crio-482d549e2ae8b4be87238c01bda8fd44fcd740d9197b3b4148e99f6ccd0acb93 WatchSource:0}: Error finding container 482d549e2ae8b4be87238c01bda8fd44fcd740d9197b3b4148e99f6ccd0acb93: Status 404 returned error can't find the container with id 482d549e2ae8b4be87238c01bda8fd44fcd740d9197b3b4148e99f6ccd0acb93
Jul 30 17:43:09.912049 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:09.912015    2569 generic.go:356] "Generic (PLEG): container finished" podID="a6e998ea-a369-4a9e-bce7-fe64925cb788" containerID="e4186618dbb4a53d84c05c59cfa176004ee42007eb4855d28d5e109d8b7a5aca" exitCode=0
Jul 30 17:43:09.912347 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:09.912103    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz" event={"ID":"a6e998ea-a369-4a9e-bce7-fe64925cb788","Type":"ContainerDied","Data":"e4186618dbb4a53d84c05c59cfa176004ee42007eb4855d28d5e109d8b7a5aca"}
Jul 30 17:43:09.912347 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:09.912180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz" event={"ID":"a6e998ea-a369-4a9e-bce7-fe64925cb788","Type":"ContainerStarted","Data":"482d549e2ae8b4be87238c01bda8fd44fcd740d9197b3b4148e99f6ccd0acb93"}
Jul 30 17:43:11.057397 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:11.057371    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"
Jul 30 17:43:11.173152 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:11.173072    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e998ea-a369-4a9e-bce7-fe64925cb788-maas-api-service-ca\") pod \"a6e998ea-a369-4a9e-bce7-fe64925cb788\" (UID: \"a6e998ea-a369-4a9e-bce7-fe64925cb788\") "
Jul 30 17:43:11.173405 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:11.173383    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e998ea-a369-4a9e-bce7-fe64925cb788-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a6e998ea-a369-4a9e-bce7-fe64925cb788" (UID: "a6e998ea-a369-4a9e-bce7-fe64925cb788"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:43:11.274353 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:11.274324    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6e998ea-a369-4a9e-bce7-fe64925cb788-maas-api-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:43:11.923655 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:11.923621    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"
Jul 30 17:43:11.923655 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:11.923643    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz" event={"ID":"a6e998ea-a369-4a9e-bce7-fe64925cb788","Type":"ContainerDied","Data":"482d549e2ae8b4be87238c01bda8fd44fcd740d9197b3b4148e99f6ccd0acb93"}
Jul 30 17:43:11.923886 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:11.923728    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="482d549e2ae8b4be87238c01bda8fd44fcd740d9197b3b4148e99f6ccd0acb93"
Jul 30 17:43:19.089307 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.089277    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"]
Jul 30 17:43:19.089850 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.089510    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf" podUID="ea41a796-3d52-4fc9-9b4b-1643f92a474f" containerName="maas-api" containerID="cri-o://517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1" gracePeriod=30
Jul 30 17:43:19.284827 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.284643    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"]
Jul 30 17:43:19.286814 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.286780    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-27deb9-pxmpz"]
Jul 30 17:43:19.334227 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.334207    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:43:19.442053 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.442027    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea41a796-3d52-4fc9-9b4b-1643f92a474f-maas-api-tls\") pod \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") "
Jul 30 17:43:19.442204 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.442093    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twps8\" (UniqueName: \"kubernetes.io/projected/ea41a796-3d52-4fc9-9b4b-1643f92a474f-kube-api-access-twps8\") pod \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") "
Jul 30 17:43:19.442294 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.442276    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea41a796-3d52-4fc9-9b4b-1643f92a474f-openshift-service-ca\") pod \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\" (UID: \"ea41a796-3d52-4fc9-9b4b-1643f92a474f\") "
Jul 30 17:43:19.442576 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.442554    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea41a796-3d52-4fc9-9b4b-1643f92a474f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea41a796-3d52-4fc9-9b4b-1643f92a474f" (UID: "ea41a796-3d52-4fc9-9b4b-1643f92a474f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:43:19.444058 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.444031    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea41a796-3d52-4fc9-9b4b-1643f92a474f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea41a796-3d52-4fc9-9b4b-1643f92a474f" (UID: "ea41a796-3d52-4fc9-9b4b-1643f92a474f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:43:19.444150 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.444093    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea41a796-3d52-4fc9-9b4b-1643f92a474f-kube-api-access-twps8" (OuterVolumeSpecName: "kube-api-access-twps8") pod "ea41a796-3d52-4fc9-9b4b-1643f92a474f" (UID: "ea41a796-3d52-4fc9-9b4b-1643f92a474f"). InnerVolumeSpecName "kube-api-access-twps8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:43:19.543381 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.543354    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea41a796-3d52-4fc9-9b4b-1643f92a474f-openshift-service-ca\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:43:19.543381 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.543381    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea41a796-3d52-4fc9-9b4b-1643f92a474f-maas-api-tls\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:43:19.543527 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.543402    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twps8\" (UniqueName: \"kubernetes.io/projected/ea41a796-3d52-4fc9-9b4b-1643f92a474f-kube-api-access-twps8\") on node \"ip-10-0-135-111.ec2.internal\" DevicePath \"\""
Jul 30 17:43:19.965237 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.965199    2569 generic.go:356] "Generic (PLEG): container finished" podID="ea41a796-3d52-4fc9-9b4b-1643f92a474f" containerID="517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1" exitCode=0
Jul 30 17:43:19.965373 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.965256    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"
Jul 30 17:43:19.965373 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.965274    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf" event={"ID":"ea41a796-3d52-4fc9-9b4b-1643f92a474f","Type":"ContainerDied","Data":"517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1"}
Jul 30 17:43:19.965373 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.965330    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf" event={"ID":"ea41a796-3d52-4fc9-9b4b-1643f92a474f","Type":"ContainerDied","Data":"59a35737e1ad8984ed31c67898db883dfaf1c40491a2c6fedcbc3d60229ebb8a"}
Jul 30 17:43:19.965373 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.965361    2569 scope.go:117] "RemoveContainer" containerID="517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1"
Jul 30 17:43:19.975424 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.975404    2569 scope.go:117] "RemoveContainer" containerID="517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1"
Jul 30 17:43:19.975731 ip-10-0-135-111 kubenswrapper[2569]: E0730 17:43:19.975708    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1\": container with ID starting with 517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1 not found: ID does not exist" containerID="517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1"
Jul 30 17:43:19.975804 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.975742    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1"} err="failed to get container status \"517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1\": rpc error: code = NotFound desc = could not find container \"517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1\": container with ID starting with 517710e9bffa1528c935473fbaf732dadf2df9d90150947df0fdfeb73aeeb5f1 not found: ID does not exist"
Jul 30 17:43:19.988065 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.988032    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"]
Jul 30 17:43:19.993639 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:19.993597    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27deb9-5886bb4c65-rs7qf"]
Jul 30 17:43:21.000552 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:21.000517    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6e998ea-a369-4a9e-bce7-fe64925cb788" path="/var/lib/kubelet/pods/a6e998ea-a369-4a9e-bce7-fe64925cb788/volumes"
Jul 30 17:43:21.000956 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:21.000927    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea41a796-3d52-4fc9-9b4b-1643f92a474f" path="/var/lib/kubelet/pods/ea41a796-3d52-4fc9-9b4b-1643f92a474f/volumes"
Jul 30 17:43:32.228235 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:32.228188    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f6d974979-cdhxs_c228f2f1-c36f-4bc3-ac18-509574700266/manager/0.log"
Jul 30 17:43:32.716546 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:32.716522    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bfff65bb6-dxns8_238cc14e-586d-4938-8b5a-c9f2d0cce2e8/copy-manifests/0.log"
Jul 30 17:43:32.724182 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:32.724159    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bfff65bb6-dxns8_238cc14e-586d-4938-8b5a-c9f2d0cce2e8/manager/0.log"
Jul 30 17:43:33.233726 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:33.233699    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-ff4449fc7-mm6cq_b36c00ab-ef8b-4328-a963-624cc8b1c38a/maas-api/0.log"
Jul 30 17:43:33.564659 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:33.564591    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757210-nbqr4_88ed3121-f6ab-4249-bd48-8be02f38c252/cleanup/0.log"
Jul 30 17:43:33.676179 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:33.676155    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-bfmmb_408fbd2d-5ac5-46c8-90c4-ca336153dbb8/postgres/0.log"
Jul 30 17:43:35.378987 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:35.378922    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mnzv2_7c7a6f57-015e-4a1f-9a1e-04c4dfb73905/kuadrant-console-plugin/0.log"
Jul 30 17:43:36.573192 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:36.573161    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbc57f5f6-wbpzm_9d99c44c-128d-4fe8-8d3d-a8d655e18cd9/kube-auth-proxy/0.log"
Jul 30 17:43:36.821355 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:36.821324    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-m87vz_9093bfce-5746-4229-817f-3ceabe0b6b5e/payload-pre-processing/0.log"
Jul 30 17:43:36.943287 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:36.943265    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wgxkq_1d438334-8a53-4796-8080-8ce8dabe3afc/payload-processing/0.log"
Jul 30 17:43:37.542282 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:37.542249    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-8slw6_c3febbc4-7bac-461a-b52f-c4edb22e2b36/main/0.log"
Jul 30 17:43:37.879760 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:37.879690    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-wbqvj_a853848e-1449-4ae2-834b-912322c73848/main/0.log"
Jul 30 17:43:46.963134 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:46.963107    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bwspl_7db173ed-c34e-43b4-9251-835908a35f70/global-pull-secret-syncer/0.log"
Jul 30 17:43:47.164692 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:47.164664    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rx4xb_8a35786b-a78a-4291-ba65-907e78441e0c/konnectivity-agent/0.log"
Jul 30 17:43:47.249591 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:47.249517    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-111.ec2.internal_3c77bfff64908136b53abdd3eaff108d/haproxy/0.log"
Jul 30 17:43:52.691061 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:52.691038    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mnzv2_7c7a6f57-015e-4a1f-9a1e-04c4dfb73905/kuadrant-console-plugin/0.log"
Jul 30 17:43:55.150891 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.150858    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-65495d8796-2pnkx_9cdb2448-daf4-41c7-88c0-3677852f671b/metrics-server/0.log"
Jul 30 17:43:55.216500 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.216476    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxvvr_648cf43a-f478-40e8-92af-40e987c5de73/node-exporter/0.log"
Jul 30 17:43:55.238559 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.238533    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxvvr_648cf43a-f478-40e8-92af-40e987c5de73/kube-rbac-proxy/0.log"
Jul 30 17:43:55.260644 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.260569    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxvvr_648cf43a-f478-40e8-92af-40e987c5de73/init-textfile/0.log"
Jul 30 17:43:55.555761 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.555708    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5be07d24-4225-4e66-abf4-2cf4536dfa4c/prometheus/0.log"
Jul 30 17:43:55.580973 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.580950    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5be07d24-4225-4e66-abf4-2cf4536dfa4c/config-reloader/0.log"
Jul 30 17:43:55.605197 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.605175    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5be07d24-4225-4e66-abf4-2cf4536dfa4c/thanos-sidecar/0.log"
Jul 30 17:43:55.626770 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.626750    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5be07d24-4225-4e66-abf4-2cf4536dfa4c/kube-rbac-proxy-web/0.log"
Jul 30 17:43:55.648051 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.648033    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5be07d24-4225-4e66-abf4-2cf4536dfa4c/kube-rbac-proxy/0.log"
Jul 30 17:43:55.671245 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.671221    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5be07d24-4225-4e66-abf4-2cf4536dfa4c/kube-rbac-proxy-thanos/0.log"
Jul 30 17:43:55.694177 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.694159    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5be07d24-4225-4e66-abf4-2cf4536dfa4c/init-config-reloader/0.log"
Jul 30 17:43:55.730179 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.730159    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-645xr_52f328e8-7b11-4103-bcd1-203f6b4bbdfe/prometheus-operator/0.log"
Jul 30 17:43:55.749248 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:55.749226    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-645xr_52f328e8-7b11-4103-bcd1-203f6b4bbdfe/kube-rbac-proxy/0.log"
Jul 30 17:43:57.252341 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.252315    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"]
Jul 30 17:43:57.252730 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.252644    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e998ea-a369-4a9e-bce7-fe64925cb788" containerName="revoke-keys"
Jul 30 17:43:57.252730 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.252654    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e998ea-a369-4a9e-bce7-fe64925cb788" containerName="revoke-keys"
Jul 30 17:43:57.252730 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.252685    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea41a796-3d52-4fc9-9b4b-1643f92a474f" containerName="maas-api"
Jul 30 17:43:57.252730 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.252691    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea41a796-3d52-4fc9-9b4b-1643f92a474f" containerName="maas-api"
Jul 30 17:43:57.252872 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.252747    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea41a796-3d52-4fc9-9b4b-1643f92a474f" containerName="maas-api"
Jul 30 17:43:57.252872 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.252754    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6e998ea-a369-4a9e-bce7-fe64925cb788" containerName="revoke-keys"
Jul 30 17:43:57.255881 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.255862    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.258831 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.258814    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pt5b5\"/\"openshift-service-ca.crt\""
Jul 30 17:43:57.260051 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.260030    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pt5b5\"/\"kube-root-ca.crt\""
Jul 30 17:43:57.260146 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.260098    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pt5b5\"/\"default-dockercfg-mvv78\""
Jul 30 17:43:57.264341 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.264319    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"]
Jul 30 17:43:57.433252 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.433224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7dn4\" (UniqueName: \"kubernetes.io/projected/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-kube-api-access-c7dn4\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.433394 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.433267    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-podres\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.433394 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.433283    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-sys\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.433394 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.433384    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-proc\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.433551 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.433460    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-lib-modules\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.534816 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.534755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-lib-modules\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.534816 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.534796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7dn4\" (UniqueName: \"kubernetes.io/projected/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-kube-api-access-c7dn4\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.534974 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.534839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-podres\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.534974 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.534861    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-sys\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.534974 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.534890    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-proc\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.534974 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.534932    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-lib-modules\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.534974 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.534957    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-proc\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.535161 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.534977    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-podres\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.535161 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.534980    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-sys\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.545564 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.545545    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7dn4\" (UniqueName: \"kubernetes.io/projected/02fadeea-fe24-4a2a-b087-d4d36dbbf8f6-kube-api-access-c7dn4\") pod \"perf-node-gather-daemonset-vsjgw\" (UID: \"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6\") " pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.566459 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.566437    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:43:57.693021 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:57.692994    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"]
Jul 30 17:43:57.694461 ip-10-0-135-111 kubenswrapper[2569]: W0730 17:43:57.694432    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod02fadeea_fe24_4a2a_b087_d4d36dbbf8f6.slice/crio-9d9394e9b5dc671fdeb233f11ab572cb28177df79942c0c3a7312839af401e24 WatchSource:0}: Error finding container 9d9394e9b5dc671fdeb233f11ab572cb28177df79942c0c3a7312839af401e24: Status 404 returned error can't find the container with id 9d9394e9b5dc671fdeb233f11ab572cb28177df79942c0c3a7312839af401e24
Jul 30 17:43:58.149230 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:58.149192    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw" event={"ID":"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6","Type":"ContainerStarted","Data":"42f71c4537b6e165230415b31e574aed3d05f82befeb5e3b6d9e0326e4f16d84"}
Jul 30 17:43:58.149410 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:58.149326    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw" event={"ID":"02fadeea-fe24-4a2a-b087-d4d36dbbf8f6","Type":"ContainerStarted","Data":"9d9394e9b5dc671fdeb233f11ab572cb28177df79942c0c3a7312839af401e24"}
Jul 30 17:43:58.168254 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:58.168207    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw" podStartSLOduration=1.168195274 podStartE2EDuration="1.168195274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:43:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:43:58.166553658 +0000 UTC m=+4065.768981283" watchObservedRunningTime="2026-07-30 17:43:58.168195274 +0000 UTC m=+4065.770622900"
Jul 30 17:43:58.372121 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:58.372093    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-666c94bb4b-ttns4_dd427e00-b844-4128-86b0-bb12d0c26bd2/console/0.log"
Jul 30 17:43:58.416446 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:58.416389    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-ltd5p_5e9801d4-cabf-4d45-9526-f002e1451590/download-server/0.log"
Jul 30 17:43:59.153246 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:43:59.153218    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:44:00.057287 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:00.057264    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c68q5_f4cb5628-b926-4f78-bfe1-c58a1850b600/dns/0.log"
Jul 30 17:44:00.079243 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:00.079222    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c68q5_f4cb5628-b926-4f78-bfe1-c58a1850b600/kube-rbac-proxy/0.log"
Jul 30 17:44:00.152875 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:00.152855    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-79c74_264e7227-dd74-4d87-9f65-107983fb8538/dns-node-resolver/0.log"
Jul 30 17:44:00.741196 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:00.741169    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54b44df6c6-6dkrn_e1d983da-cbef-4822-8d92-f9b9accdfb4b/registry/0.log"
Jul 30 17:44:00.766998 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:00.766973    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kw2nz_0c926555-bc21-4e17-98de-15f8d25b951a/node-ca/0.log"
Jul 30 17:44:01.954989 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:01.954961    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbc57f5f6-wbpzm_9d99c44c-128d-4fe8-8d3d-a8d655e18cd9/kube-auth-proxy/0.log"
Jul 30 17:44:02.049328 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:02.049304    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-m87vz_9093bfce-5746-4229-817f-3ceabe0b6b5e/payload-pre-processing/0.log"
Jul 30 17:44:02.088850 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:02.088811    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wgxkq_1d438334-8a53-4796-8080-8ce8dabe3afc/payload-processing/0.log"
Jul 30 17:44:02.942649 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:02.942593    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kj7pk_7a2989d8-f863-4d75-8724-8a68cce35870/serve-healthcheck-canary/0.log"
Jul 30 17:44:03.475535 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:03.475504    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-khc6x_d11a3a2b-9176-4dc1-ad78-6b1002aa5947/insights-operator/0.log"
Jul 30 17:44:03.476810 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:03.476784    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-khc6x_d11a3a2b-9176-4dc1-ad78-6b1002aa5947/insights-operator/1.log"
Jul 30 17:44:03.499961 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:03.499944    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-74z6w_a31939ac-a41d-44a3-b3ee-853c40acf93f/kube-rbac-proxy/0.log"
Jul 30 17:44:03.526435 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:03.526414    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-74z6w_a31939ac-a41d-44a3-b3ee-853c40acf93f/exporter/0.log"
Jul 30 17:44:03.548201 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:03.548174    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-74z6w_a31939ac-a41d-44a3-b3ee-853c40acf93f/extractor/0.log"
Jul 30 17:44:05.166216 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:05.166185    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pt5b5/perf-node-gather-daemonset-vsjgw"
Jul 30 17:44:06.594824 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:06.594787    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f6d974979-cdhxs_c228f2f1-c36f-4bc3-ac18-509574700266/manager/0.log"
Jul 30 17:44:06.838591 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:06.838505    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bfff65bb6-dxns8_238cc14e-586d-4938-8b5a-c9f2d0cce2e8/manager/0.log"
Jul 30 17:44:06.860319 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:06.860296    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bfff65bb6-dxns8_238cc14e-586d-4938-8b5a-c9f2d0cce2e8/copy-manifests/0.log"
Jul 30 17:44:08.135857 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:08.135768    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbc57f5f6-wbpzm_9d99c44c-128d-4fe8-8d3d-a8d655e18cd9/kube-auth-proxy/0.log"
Jul 30 17:44:08.207391 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:08.207360    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-m87vz_9093bfce-5746-4229-817f-3ceabe0b6b5e/payload-pre-processing/0.log"
Jul 30 17:44:08.235175 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:08.235154    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wgxkq_1d438334-8a53-4796-8080-8ce8dabe3afc/payload-processing/0.log"
Jul 30 17:44:08.256400 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:08.256377    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b8dffc46b-98kjc_b1bba25f-b4f4-4130-9822-96c0b8d19408/manager/0.log"
Jul 30 17:44:09.830038 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:09.830005    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-bbc57f5f6-wbpzm_9d99c44c-128d-4fe8-8d3d-a8d655e18cd9/kube-auth-proxy/0.log"
Jul 30 17:44:09.909077 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:09.909049    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-m87vz_9093bfce-5746-4229-817f-3ceabe0b6b5e/payload-pre-processing/0.log"
Jul 30 17:44:09.940080 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:09.940055    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-wgxkq_1d438334-8a53-4796-8080-8ce8dabe3afc/payload-processing/0.log"
Jul 30 17:44:13.333839 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:13.333805    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-v6prq_213879ae-ddf4-40a6-bfe7-a24265d53411/migrator/0.log"
Jul 30 17:44:13.356701 ip-10-0-135-111 kubenswrapper[2569]: I0730 17:44:13.356679    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-v6prq_213879ae-ddf4-40a6-bfe7-a24265d53411/graceful-termination/0.log"